2023-06-23T17:50:38.245 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T17:50:38.247 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --archive /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866 --owner scheduled_rishabh@teuthology --verbose --name rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi -- /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866/orig.config.yaml 2023-06-23T17:50:38.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T17:50:38.379 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866 branch: wip-rishabh-improvements-authmon description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7313866' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-improvements-authmon ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-improvements-authmon sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-improvements-authmon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-23_17:37:30 tube: smithi user: rishabh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T17:50:38.380 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa; will attempt to use it 2023-06-23T17:50:38.381 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks 2023-06-23T17:50:38.381 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T17:50:38.383 INFO:teuthology.task.internal:Checking packages... 2023-06-23T17:50:38.412 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8627af3e0adcb765a3c249fcc209cba9f4873e1b' 2023-06-23T17:50:38.412 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T17:50:38.413 INFO:teuthology.packaging:ref: None 2023-06-23T17:50:38.413 INFO:teuthology.packaging:tag: None 2023-06-23T17:50:38.413 INFO:teuthology.packaging:branch: wip-rishabh-improvements-authmon 2023-06-23T17:50:38.413 INFO:teuthology.packaging:sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T17:50:38.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-improvements-authmon 2023-06-23T17:50:38.549 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4479-g8627af3e-1focal 2023-06-23T17:50:38.550 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T17:50:38.557 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T17:50:38.557 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T17:50:38.607 INFO:teuthology.task.internal:Saving configuration 2023-06-23T17:50:38.622 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T17:50:38.697 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T17:50:38.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:44:17.714009', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-06-23T17:50:38.751 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:44:17.715604', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-06-23T17:50:38.752 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T17:50:38.761 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-23T17:50:38.761 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-23T17:50:38.761 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T17:50:38.843 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc68bcab040>, signals=[15]) 2023-06-23T17:50:38.845 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T17:50:38.852 INFO:teuthology.task.internal:Opening connections... 2023-06-23T17:50:38.852 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T17:50:38.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:50:38.909 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T17:50:38.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:50:38.955 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T17:50:38.964 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2023-06-23T17:50:38.993 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2023-06-23T17:50:38.993 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-06-23T17:50:39.043 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu" 2023-06-23T17:50:39.044 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-06-23T17:50:39.044 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu 2023-06-23T17:50:39.045 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian 2023-06-23T17:50:39.045 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-06-23T17:50:39.045 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="20.04" 2023-06-23T17:50:39.046 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/" 2023-06-23T17:50:39.046 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-06-23T17:50:39.046 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-06-23T17:50:39.046 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-06-23T17:50:39.047 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=focal 2023-06-23T17:50:39.047 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=focal 2023-06-23T17:50:39.048 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-06-23T17:50:39.078 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-06-23T17:50:39.086 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-06-23T17:50:39.087 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-06-23T17:50:39.134 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2023-06-23T17:50:39.135 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-06-23T17:50:39.135 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2023-06-23T17:50:39.135 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2023-06-23T17:50:39.135 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="20.04" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-06-23T17:50:39.136 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=focal 2023-06-23T17:50:39.137 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=focal 2023-06-23T17:50:39.137 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-06-23T17:50:39.161 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T17:50:39.170 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T17:50:39.175 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T17:50:39.176 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2023-06-23T17:50:39.178 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2023-06-23T17:50:39.184 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T17:50:39.190 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T17:50:39.190 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2023-06-23T17:50:39.193 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2023-06-23T17:50:39.238 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T17:50:39.298 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T17:50:39.314 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-23T17:50:39.315 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-23T17:50:39.316 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T17:50:39.317 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T17:50:39.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T17:50:39.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T17:50:39.323 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-06-23T17:50:39.323 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-06-23T17:50:39.329 INFO:teuthology.orchestra.run.smithi169.stdout:5.4.0-124-generic 2023-06-23T17:50:39.329 INFO:teuthology.task.kernel:Running kernel on smithi169: 5.4.0-124-generic 2023-06-23T17:50:39.329 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get clean 2023-06-23T17:50:39.330 INFO:teuthology.orchestra.run.smithi022.stdout:5.4.0-124-generic 2023-06-23T17:50:39.331 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.4.0-124-generic 2023-06-23T17:50:39.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2023-06-23T17:50:39.427 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get update 2023-06-23T17:50:39.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2023-06-23T17:50:39.727 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-06-23T17:50:40.201 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:50:40.241 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:50:40.242 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-06-23T17:50:40.283 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-06-23T17:50:40.285 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-06-23T17:50:40.325 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [613 kB] 2023-06-23T17:50:40.626 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-06-23T17:50:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-06-23T17:50:40.654 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,267 kB] 2023-06-23T17:50:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,267 kB] 2023-06-23T17:50:40.770 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [844 kB] 2023-06-23T17:50:40.839 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [362 kB] 2023-06-23T17:50:40.853 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.0 kB] 2023-06-23T17:50:40.854 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,885 kB] 2023-06-23T17:50:40.859 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,649 kB] 2023-06-23T17:50:40.917 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [30.3 kB] 2023-06-23T17:50:40.918 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [264 kB] 2023-06-23T17:50:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2023-06-23T17:50:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [603 kB] 2023-06-23T17:50:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [849 kB] 2023-06-23T17:50:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,649 kB] 2023-06-23T17:50:41.119 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [176 kB] 2023-06-23T17:50:41.172 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [444 kB] 2023-06-23T17:50:41.184 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.9 kB] 2023-06-23T17:50:41.185 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,994 kB] 2023-06-23T17:50:41.215 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [613 kB] 2023-06-23T17:50:41.230 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [362 kB] 2023-06-23T17:50:41.237 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [31.6 kB] 2023-06-23T17:50:41.238 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [280 kB] 2023-06-23T17:50:41.240 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.0 kB] 2023-06-23T17:50:41.240 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [30.3 kB] 2023-06-23T17:50:41.240 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,885 kB] 2023-06-23T17:50:41.242 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2023-06-23T17:50:41.310 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [264 kB] 2023-06-23T17:50:41.314 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [844 kB] 2023-06-23T17:50:41.317 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2023-06-23T17:50:41.317 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [849 kB] 2023-06-23T17:50:41.331 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [444 kB] 2023-06-23T17:50:41.338 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [603 kB] 2023-06-23T17:50:41.340 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.9 kB] 2023-06-23T17:50:41.341 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [31.6 kB] 2023-06-23T17:50:41.341 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,994 kB] 2023-06-23T17:50:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [733 kB] 2023-06-23T17:50:41.409 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [280 kB] 2023-06-23T17:50:41.414 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2023-06-23T17:50:41.515 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [176 kB] 2023-06-23T17:50:41.523 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [18.6 kB] 2023-06-23T17:50:41.529 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.9 kB] 2023-06-23T17:50:41.537 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2023-06-23T17:50:41.539 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,488 B] 2023-06-23T17:50:41.542 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2023-06-23T17:50:41.578 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [733 kB] 2023-06-23T17:50:41.923 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [18.6 kB] 2023-06-23T17:50:41.928 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.9 kB] 2023-06-23T17:50:41.937 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2023-06-23T17:50:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,488 B] 2023-06-23T17:50:41.941 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2023-06-23T17:50:41.990 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,074 kB] 2023-06-23T17:50:42.097 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [256 kB] 2023-06-23T17:50:42.118 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.1 kB] 2023-06-23T17:50:42.119 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-06-23T17:50:42.121 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] 2023-06-23T17:50:42.122 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,408 B] 2023-06-23T17:50:42.123 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [612 B] 2023-06-23T17:50:42.123 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-06-23T17:50:42.125 INFO:teuthology.orchestra.run.smithi169.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-06-23T17:50:42.130 INFO:teuthology.orchestra.run.smithi169.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-06-23T17:50:42.162 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,074 kB] 2023-06-23T17:50:42.269 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [256 kB] 2023-06-23T17:50:42.289 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.1 kB] 2023-06-23T17:50:42.292 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-06-23T17:50:42.293 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] 2023-06-23T17:50:42.294 INFO:teuthology.orchestra.run.smithi169.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-06-23T17:50:42.295 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,408 B] 2023-06-23T17:50:42.295 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [612 B] 2023-06-23T17:50:42.295 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-06-23T17:50:42.298 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-06-23T17:50:42.300 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-06-23T17:50:42.462 INFO:teuthology.orchestra.run.smithi169.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-06-23T17:50:42.464 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-06-23T17:50:42.512 INFO:teuthology.orchestra.run.smithi169.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-06-23T17:50:42.627 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-06-23T17:50:42.678 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-06-23T17:50:44.513 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 16.0 MB in 3s (5,225 kB/s) 2023-06-23T17:50:45.269 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 16.0 MB in 3s (4,583 kB/s) 2023-06-23T17:50:45.979 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:50:46.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-06-23T17:50:46.074 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:50:46.267 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-06-23T17:50:46.268 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-06-23T17:50:46.439 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2023-06-23T17:50:46.439 INFO:teuthology.orchestra.run.smithi169.stdout: linux-generic linux-headers-5.4.0-152 linux-headers-5.4.0-152-generic 2023-06-23T17:50:46.439 INFO:teuthology.orchestra.run.smithi169.stdout: linux-headers-generic linux-image-5.4.0-152-generic 2023-06-23T17:50:46.440 INFO:teuthology.orchestra.run.smithi169.stdout: linux-modules-5.4.0-152-generic linux-modules-extra-5.4.0-152-generic 2023-06-23T17:50:46.440 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2023-06-23T17:50:46.440 INFO:teuthology.orchestra.run.smithi169.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-06-23T17:50:46.441 INFO:teuthology.orchestra.run.smithi169.stdout:Recommended packages: 2023-06-23T17:50:46.441 INFO:teuthology.orchestra.run.smithi169.stdout: thermald 2023-06-23T17:50:46.498 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2023-06-23T17:50:46.498 INFO:teuthology.orchestra.run.smithi169.stdout: linux-headers-5.4.0-152 linux-headers-5.4.0-152-generic 2023-06-23T17:50:46.498 INFO:teuthology.orchestra.run.smithi169.stdout: linux-image-5.4.0-152-generic linux-modules-5.4.0-152-generic 2023-06-23T17:50:46.498 INFO:teuthology.orchestra.run.smithi169.stdout: linux-modules-extra-5.4.0-152-generic 2023-06-23T17:50:46.499 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2023-06-23T17:50:46.500 INFO:teuthology.orchestra.run.smithi169.stdout: linux-generic linux-headers-generic linux-image-generic 2023-06-23T17:50:46.552 INFO:teuthology.orchestra.run.smithi169.stdout:3 upgraded, 5 newly installed, 0 to remove and 223 not upgraded. 2023-06-23T17:50:46.552 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 77.1 MB of archives. 2023-06-23T17:50:46.553 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 380 MB of additional disk space will be used. 2023-06-23T17:50:46.553 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-152-generic amd64 5.4.0-152.169 [15.0 MB] 2023-06-23T17:50:46.715 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:50:46.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-06-23T17:50:46.826 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:50:46.871 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-152-generic amd64 5.4.0-152.169 [10.5 MB] 2023-06-23T17:50:46.964 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-152-generic amd64 5.4.0-152.169 [39.2 MB] 2023-06-23T17:50:47.028 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2023-06-23T17:50:47.029 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2023-06-23T17:50:47.209 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2023-06-23T17:50:47.210 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-5.4.0-152 linux-headers-5.4.0-152-generic 2023-06-23T17:50:47.210 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-generic linux-image-5.4.0-152-generic 2023-06-23T17:50:47.210 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-5.4.0-152-generic linux-modules-extra-5.4.0-152-generic 2023-06-23T17:50:47.210 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2023-06-23T17:50:47.211 INFO:teuthology.orchestra.run.smithi022.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-06-23T17:50:47.211 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2023-06-23T17:50:47.211 INFO:teuthology.orchestra.run.smithi022.stdout: thermald 2023-06-23T17:50:47.268 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2023-06-23T17:50:47.268 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-5.4.0-152 linux-headers-5.4.0-152-generic 2023-06-23T17:50:47.268 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-152-generic linux-modules-5.4.0-152-generic 2023-06-23T17:50:47.269 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-extra-5.4.0-152-generic 2023-06-23T17:50:47.269 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2023-06-23T17:50:47.270 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-generic linux-image-generic 2023-06-23T17:50:47.427 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.152.149 [1,900 B] 2023-06-23T17:50:47.427 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.152.149 [2,596 B] 2023-06-23T17:50:47.427 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-152 all 5.4.0-152.169 [11.0 MB] 2023-06-23T17:50:47.453 INFO:teuthology.orchestra.run.smithi022.stdout:3 upgraded, 5 newly installed, 0 to remove and 223 not upgraded. 2023-06-23T17:50:47.453 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 77.1 MB of archives. 2023-06-23T17:50:47.454 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 380 MB of additional disk space will be used. 2023-06-23T17:50:47.454 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-152-generic amd64 5.4.0-152.169 [15.0 MB] 2023-06-23T17:50:47.517 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-152-generic amd64 5.4.0-152.169 [1,362 kB] 2023-06-23T17:50:47.527 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.152.149 [2,464 B] 2023-06-23T17:50:48.222 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 77.1 MB in 1s (75.5 MB/s) 2023-06-23T17:50:48.421 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-modules-5.4.0-152-generic. 2023-06-23T17:50:48.596 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-152-generic amd64 5.4.0-152.169 [10.5 MB] 2023-06-23T17:50:48.866 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-152-generic amd64 5.4.0-152.169 [39.2 MB] 2023-06-23T17:50:50.168 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-06-23T17:50:50.173 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../0-linux-modules-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:50.215 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.152.149 [1,900 B] 2023-06-23T17:50:50.216 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.152.149 [2,596 B] 2023-06-23T17:50:50.216 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-152 all 5.4.0-152.169 [11.0 MB] 2023-06-23T17:50:50.302 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-modules-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:50.504 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-152-generic amd64 5.4.0-152.169 [1,362 kB] 2023-06-23T17:50:50.551 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.152.149 [2,464 B] 2023-06-23T17:50:51.272 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 77.1 MB in 3s (23.5 MB/s) 2023-06-23T17:50:51.473 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-5.4.0-152-generic. 2023-06-23T17:50:52.311 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-image-5.4.0-152-generic. 2023-06-23T17:50:52.325 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../1-linux-image-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:53.136 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-06-23T17:50:53.141 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-linux-modules-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:53.573 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-152-generic. 2023-06-23T17:50:53.588 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:53.631 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-modules-extra-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:55.169 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-image-5.4.0-152-generic. 2023-06-23T17:50:55.185 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-linux-image-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:55.279 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:56.405 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-152-generic. 2023-06-23T17:50:56.419 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:50:56.463 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-extra-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:50:58.847 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../3-linux-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:50:59.005 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:50:59.363 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:50:59.483 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-image-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:50:59.878 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-headers-5.4.0-152. 2023-06-23T17:50:59.895 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../5-linux-headers-5.4.0-152_5.4.0-152.169_all.deb ... 2023-06-23T17:50:59.936 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-headers-5.4.0-152 (5.4.0-152.169) ... 2023-06-23T17:51:01.456 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-linux-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:51:01.583 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:51:01.926 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:51:02.053 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:51:02.372 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.4.0-152. 2023-06-23T17:51:02.387 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-linux-headers-5.4.0-152_5.4.0-152.169_all.deb ... 2023-06-23T17:51:02.422 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.4.0-152 (5.4.0-152.169) ... 2023-06-23T17:51:05.952 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-headers-5.4.0-152-generic. 2023-06-23T17:51:05.974 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../6-linux-headers-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:51:06.048 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-headers-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:07.821 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.4.0-152-generic. 2023-06-23T17:51:07.843 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-linux-headers-5.4.0-152-generic_5.4.0-152.169_amd64.deb ... 2023-06-23T17:51:07.888 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:08.839 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:51:08.967 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-headers-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:51:09.319 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-headers-5.4.0-152 (5.4.0-152.169) ... 2023-06-23T17:51:09.472 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-headers-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:09.598 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-modules-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:10.810 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.152.149_amd64.deb ... 2023-06-23T17:51:10.852 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:10.953 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-generic (5.4.0.152.149) over (5.4.0.124.125) ... 2023-06-23T17:51:11.273 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.4.0-152 (5.4.0-152.169) ... 2023-06-23T17:51:11.450 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:11.576 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:12.010 INFO:teuthology.orchestra.run.smithi169.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:12.011 INFO:teuthology.orchestra.run.smithi169.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:12.155 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-headers-generic (5.4.0.152.149) ... 2023-06-23T17:51:12.306 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-modules-extra-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:12.845 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:13.499 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-image-generic (5.4.0.152.149) ... 2023-06-23T17:51:13.625 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-generic (5.4.0.152.149) ... 2023-06-23T17:51:13.768 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:13.842 INFO:teuthology.orchestra.run.smithi169.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-06-23T17:51:13.843 INFO:teuthology.orchestra.run.smithi169.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:14.285 INFO:teuthology.orchestra.run.smithi022.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:14.285 INFO:teuthology.orchestra.run.smithi022.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:15.331 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-generic (5.4.0.152.149) ... 2023-06-23T17:51:15.465 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-extra-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:16.717 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-generic (5.4.0.152.149) ... 2023-06-23T17:51:16.843 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-generic (5.4.0.152.149) ... 2023-06-23T17:51:16.995 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for linux-image-5.4.0-152-generic (5.4.0-152.169) ... 2023-06-23T17:51:17.062 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-06-23T17:51:17.062 INFO:teuthology.orchestra.run.smithi022.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:36.121 INFO:teuthology.orchestra.run.smithi169.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-06-23T17:51:36.121 INFO:teuthology.orchestra.run.smithi169.stdout:Sourcing file `/etc/default/grub' 2023-06-23T17:51:36.133 INFO:teuthology.orchestra.run.smithi169.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-06-23T17:51:36.144 INFO:teuthology.orchestra.run.smithi169.stdout:Generating grub configuration file ... 2023-06-23T17:51:36.532 INFO:teuthology.orchestra.run.smithi169.stdout:Found linux image: /boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:36.561 INFO:teuthology.orchestra.run.smithi169.stdout:Found initrd image: /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:36.962 INFO:teuthology.orchestra.run.smithi169.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-06-23T17:51:36.985 INFO:teuthology.orchestra.run.smithi169.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-06-23T17:51:39.187 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-06-23T17:51:39.187 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub' 2023-06-23T17:51:39.199 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-06-23T17:51:39.213 INFO:teuthology.orchestra.run.smithi022.stdout:Generating grub configuration file ... 2023-06-23T17:51:39.541 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:39.563 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:39.631 INFO:teuthology.orchestra.run.smithi169.stdout:done 2023-06-23T17:51:39.931 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-06-23T17:51:39.942 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-06-23T17:51:40.070 DEBUG:teuthology.orchestra.run.smithi169:> dpkg -s linux-image-generic 2023-06-23T17:51:40.114 INFO:teuthology.orchestra.run.smithi169.stdout:Package: linux-image-generic 2023-06-23T17:51:40.114 INFO:teuthology.orchestra.run.smithi169.stdout:Status: install ok installed 2023-06-23T17:51:40.114 INFO:teuthology.orchestra.run.smithi169.stdout:Priority: optional 2023-06-23T17:51:40.114 INFO:teuthology.orchestra.run.smithi169.stdout:Section: kernel 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Installed-Size: 19 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Maintainer: Ubuntu Kernel Team 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: amd64 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Source: linux-meta 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Version: 5.4.0.152.149 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-06-23T17:51:40.115 INFO:teuthology.orchestra.run.smithi169.stdout:Depends: linux-image-5.4.0-152-generic, linux-modules-extra-5.4.0-152-generic, linux-firmware, intel-microcode, amd64-microcode 2023-06-23T17:51:40.116 INFO:teuthology.orchestra.run.smithi169.stdout:Recommends: thermald 2023-06-23T17:51:40.116 INFO:teuthology.orchestra.run.smithi169.stdout:Description: Generic Linux kernel image 2023-06-23T17:51:40.116 INFO:teuthology.orchestra.run.smithi169.stdout: This package will always depend on the latest generic kernel image 2023-06-23T17:51:40.116 INFO:teuthology.orchestra.run.smithi169.stdout: available. 2023-06-23T17:51:40.116 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-152-generic 2023-06-23T17:51:40.117 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T17:51:40.117 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T17:51:40.117 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-06-23T17:51:40.117 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T17:51:40.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get clean 2023-06-23T17:51:40.197 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get update 2023-06-23T17:51:40.326 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-06-23T17:51:40.816 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:51:40.833 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-06-23T17:51:40.850 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-06-23T17:51:42.361 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:51:42.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-06-23T17:51:42.452 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:51:42.611 INFO:teuthology.orchestra.run.smithi022.stdout:done 2023-06-23T17:51:42.646 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-06-23T17:51:42.646 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-06-23T17:51:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:linux-image-generic is already the newest version (5.4.0.152.149). 2023-06-23T17:51:42.857 INFO:teuthology.orchestra.run.smithi169.stdout:0 upgraded, 0 newly installed, 0 to remove and 223 not upgraded. 2023-06-23T17:51:42.858 DEBUG:teuthology.orchestra.run.smithi169:> dpkg -s linux-image-generic 2023-06-23T17:51:42.878 INFO:teuthology.orchestra.run.smithi169.stdout:Package: linux-image-generic 2023-06-23T17:51:42.878 INFO:teuthology.orchestra.run.smithi169.stdout:Status: install ok installed 2023-06-23T17:51:42.878 INFO:teuthology.orchestra.run.smithi169.stdout:Priority: optional 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Section: kernel 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Installed-Size: 19 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Maintainer: Ubuntu Kernel Team 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: amd64 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Source: linux-meta 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Version: 5.4.0.152.149 2023-06-23T17:51:42.879 INFO:teuthology.orchestra.run.smithi169.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-06-23T17:51:42.880 INFO:teuthology.orchestra.run.smithi169.stdout:Depends: linux-image-5.4.0-152-generic, linux-modules-extra-5.4.0-152-generic, linux-firmware, intel-microcode, amd64-microcode 2023-06-23T17:51:42.880 INFO:teuthology.orchestra.run.smithi169.stdout:Recommends: thermald 2023-06-23T17:51:42.880 INFO:teuthology.orchestra.run.smithi169.stdout:Description: Generic Linux kernel image 2023-06-23T17:51:42.880 INFO:teuthology.orchestra.run.smithi169.stdout: This package will always depend on the latest generic kernel image 2023-06-23T17:51:42.880 INFO:teuthology.orchestra.run.smithi169.stdout: available. 2023-06-23T17:51:42.880 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-06-23T17:51:42.885 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.rbuiM911FG 2023-06-23T17:51:42.885 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rbuiM911FG 2023-06-23T17:51:42.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.rbuiM911FG 2023-06-23T17:51:42.991 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.rbuiM911FG is 9KB 2023-06-23T17:51:43.003 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.rbuiM911FG 2023-06-23T17:51:43.009 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-06-23T17:51:43.015 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2023-06-23T17:51:43.067 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2023-06-23T17:51:43.067 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 19 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2023-06-23T17:51:43.068 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.4.0.152.149 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.4.0-152-generic, linux-modules-extra-5.4.0-152-generic, linux-firmware, intel-microcode, amd64-microcode 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2023-06-23T17:51:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2023-06-23T17:51:43.070 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-152-generic 2023-06-23T17:51:43.070 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T17:51:43.070 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T17:51:43.070 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-06-23T17:51:43.070 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T17:51:43.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2023-06-23T17:51:43.071 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T17:51:43.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-06-23T17:51:43.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-06-23T17:51:43.103 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-152-generic 2023-06-23T17:51:43.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo update-grub 2023-06-23T17:51:43.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2023-06-23T17:51:43.271 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-06-23T17:51:43.389 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:51:43.471 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-06-23T17:51:43.553 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-06-23T17:51:44.498 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub' 2023-06-23T17:51:44.505 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-06-23T17:51:44.517 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2023-06-23T17:51:44.794 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:44.806 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:45.031 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:51:45.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-06-23T17:51:45.127 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:51:45.201 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-06-23T17:51:45.213 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-06-23T17:51:45.307 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2023-06-23T17:51:45.307 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2023-06-23T17:51:45.516 INFO:teuthology.orchestra.run.smithi022.stdout:linux-image-generic is already the newest version (5.4.0.152.149). 2023-06-23T17:51:45.517 INFO:teuthology.orchestra.run.smithi022.stdout:0 upgraded, 0 newly installed, 0 to remove and 223 not upgraded. 2023-06-23T17:51:45.518 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2023-06-23T17:51:45.539 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2023-06-23T17:51:45.539 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2023-06-23T17:51:45.539 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2023-06-23T17:51:45.539 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 19 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.4.0.152.149 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-06-23T17:51:45.540 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.4.0-152-generic, linux-modules-extra-5.4.0-152-generic, linux-firmware, intel-microcode, amd64-microcode 2023-06-23T17:51:45.541 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2023-06-23T17:51:45.541 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2023-06-23T17:51:45.541 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2023-06-23T17:51:45.541 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2023-06-23T17:51:45.541 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2023-06-23T17:51:45.546 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.UcTnWFvX7k 2023-06-23T17:51:45.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub/grub.cfg /tmp/tmp.UcTnWFvX7k 2023-06-23T17:51:45.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.UcTnWFvX7k 2023-06-23T17:51:45.599 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.UcTnWFvX7k is 9KB 2023-06-23T17:51:45.612 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.UcTnWFvX7k 2023-06-23T17:51:45.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-06-23T17:51:45.676 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T17:51:45.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-06-23T17:51:45.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-06-23T17:51:45.708 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-152-generic 2023-06-23T17:51:45.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo update-grub 2023-06-23T17:51:46.873 INFO:teuthology.orchestra.run.smithi169.stderr:done 2023-06-23T17:51:46.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2023-06-23T17:51:47.025 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub' 2023-06-23T17:51:47.037 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-06-23T17:51:47.050 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2023-06-23T17:51:47.319 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.4.0-152-generic 2023-06-23T17:51:47.331 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.4.0-152-generic 2023-06-23T17:51:47.732 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-06-23T17:51:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-06-23T17:51:49.422 INFO:teuthology.orchestra.run.smithi022.stderr:done 2023-06-23T17:51:49.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2023-06-23T17:52:16.904 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-06-23T17:52:16.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-06-23T17:52:16.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:52:19.428 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-06-23T17:52:19.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-06-23T17:52:19.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:52:51.568 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-06-23T17:53:19.456 DEBUG:teuthology.orchestra.remote:timed out 2023-06-23T17:53:21.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-06-23T17:53:21.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:53:37.106 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-06-23T17:53:37.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-06-23T17:53:37.738 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-152-generic"... 2023-06-23T17:53:37.739 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-06-23T17:53:37.745 INFO:teuthology.orchestra.run.smithi169.stdout:5.4.0-152-generic 2023-06-23T17:53:37.745 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-152-generic vs 5.4.0-152-generic 2023-06-23T17:53:37.745 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T17:53:37.746 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-06-23T17:53:38.746 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-23T17:53:38.747 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T17:53:38.842 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2023-06-23T17:53:38.863 DEBUG:teuthology.parallel:result is None 2023-06-23T17:53:49.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-06-23T17:53:49.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:53:49.655 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-06-23T17:53:50.291 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-06-23T17:53:50.291 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-152-generic"... 2023-06-23T17:53:50.292 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-06-23T17:53:50.298 INFO:teuthology.orchestra.run.smithi022.stdout:5.4.0-152-generic 2023-06-23T17:53:50.298 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-152-generic vs 5.4.0-152-generic 2023-06-23T17:53:50.299 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T17:53:50.299 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-06-23T17:53:51.299 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-23T17:53:51.300 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T17:53:51.384 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2023-06-23T17:53:51.406 DEBUG:teuthology.parallel:result is None 2023-06-23T17:53:51.407 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T17:53:51.415 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T17:53:51.415 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T17:53:51.418 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T17:53:51.425 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T17:53:51.432 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T17:53:51.442 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T17:53:51.443 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T17:53:51.445 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T17:53:51.469 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T17:53:51.479 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T17:53:51.480 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T17:53:51.497 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T17:53:51.512 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T17:53:51.523 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T17:53:51.524 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T17:53:51.534 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T17:53:51.536 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T17:53:51.543 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T17:53:51.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T17:53:51.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T17:53:51.594 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T17:53:51.603 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T17:53:51.603 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T17:53:51.605 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T17:53:51.612 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T17:53:51.658 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T17:53:51.665 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T17:53:51.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T17:53:51.687 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T17:53:51.694 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T17:53:51.744 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T17:53:51.744 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T17:53:51.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2023-06-23T17:53:51.766 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2023-06-23T17:53:51.817 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T17:53:51.824 INFO:teuthology.task.internal:Starting timer... 2023-06-23T17:53:51.825 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T17:53:51.835 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T17:53:51.844 DEBUG:teuthology.task.selinux:Excluding smithi022: OS 'ubuntu' does not support SELinux 2023-06-23T17:53:51.844 DEBUG:teuthology.task.selinux:Excluding smithi169: OS 'ubuntu' does not support SELinux 2023-06-23T17:53:51.845 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T17:53:51.845 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-06-23T17:53:51.845 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T17:53:51.845 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T17:53:51.853 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-23T17:53:51.854 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T17:53:51.876 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T17:53:51.878 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T17:59:35.281 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2023-06-23T17:59:35.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-06-23T17:59:35.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:59:35.333 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-06-23T17:59:35.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-06-23T17:59:35.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T17:59:35.511 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-06-23T17:59:35.629 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T17:59:35.643 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T17:59:35.644 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T17:59:35.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T17:59:35.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T17:59:35.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T17:59:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-06-23T17:59:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Command line: ntpd -gq 2023-06-23T17:59:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: proto: precision = 0.072 usec (-24) 2023-06-23T17:59:35.681 INFO:teuthology.orchestra.run.smithi022.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-06-23T17:59:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-06-23T17:59:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: restrict ::: KOD does nothing without LIMITED. 2023-06-23T17:59:35.682 INFO:teuthology.orchestra.run.smithi022.stderr:restrict ::: KOD does nothing without LIMITED. 2023-06-23T17:59:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen and drop on 0 v6wildcard [::]:123 2023-06-23T17:59:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-06-23T17:59:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen normally on 2 lo 127.0.0.1:123 2023-06-23T17:59:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen normally on 3 ens1f0 172.21.15.22:123 2023-06-23T17:59:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen normally on 4 lo [::1]:123 2023-06-23T17:59:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:5de4%4]:123 2023-06-23T17:59:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:35 ntpd[22165]: Listening on routing socket on fd #22 for interface updates 2023-06-23T17:59:35.685 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-06-23T17:59:35.685 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Command line: ntpd -gq 2023-06-23T17:59:35.685 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: proto: precision = 0.100 usec (-23) 2023-06-23T17:59:35.685 INFO:teuthology.orchestra.run.smithi169.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-06-23T17:59:35.686 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-06-23T17:59:35.686 INFO:teuthology.orchestra.run.smithi169.stderr:restrict ::: KOD does nothing without LIMITED. 2023-06-23T17:59:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: restrict ::: KOD does nothing without LIMITED. 2023-06-23T17:59:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen and drop on 0 v6wildcard [::]:123 2023-06-23T17:59:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-06-23T17:59:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen normally on 2 lo 127.0.0.1:123 2023-06-23T17:59:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen normally on 3 enp3s0f1 172.21.15.169:123 2023-06-23T17:59:35.688 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen normally on 4 lo [::1]:123 2023-06-23T17:59:35.688 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f4d%5]:123 2023-06-23T17:59:35.688 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:35 ntpd[22271]: Listening on routing socket on fd #22 for interface updates 2023-06-23T17:59:36.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:36.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:36.688 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:36.688 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:36 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:37.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:37.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:37.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:37.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:37.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:37.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:37 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:37.683 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:37 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:37.684 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:37 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:38 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:38.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:38.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:38.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:38.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:38.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:38 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:39 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:40 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:40.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:40.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:40.688 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:40 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:41.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:41.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:41.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:41.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:41.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:41 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:42.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:42.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:42 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:42.684 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.684 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:42.685 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:42.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:42.686 INFO:teuthology.orchestra.run.smithi169.stdout:23 Jun 17:59:42 ntpd[22271]: ntpd: time slew -0.007294 s 2023-06-23T17:59:42.686 INFO:teuthology.orchestra.run.smithi169.stdout:ntpd: time slew -0.007294s 2023-06-23T17:59:42.686 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-06-23T17:59:42.687 INFO:teuthology.orchestra.run.smithi169.stderr:23 Jun 17:59:42 ntpd[22271]: can't open /var/log/ntpstats/loopstats.20230623: Permission denied 2023-06-23T17:59:42.759 INFO:teuthology.orchestra.run.smithi169.stdout: remote refid st t when poll reach delay offset jitter 2023-06-23T17:59:42.759 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================== 2023-06-23T17:59:42.759 INFO:teuthology.orchestra.run.smithi169.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:42.759 INFO:teuthology.orchestra.run.smithi169.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:42.760 INFO:teuthology.orchestra.run.smithi169.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:42.760 INFO:teuthology.orchestra.run.smithi169.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:43.681 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:43.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:43.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:43.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:43.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:43.682 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:43.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:43.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:43.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-06-23T17:59:43.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/rawstats.20230623: Permission denied 2023-06-23T17:59:43.683 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-06-23T17:59:43.684 INFO:teuthology.orchestra.run.smithi022.stdout:23 Jun 17:59:43 ntpd[22165]: ntpd: time slew +0.013463 s 2023-06-23T17:59:43.684 INFO:teuthology.orchestra.run.smithi022.stdout:ntpd: time slew +0.013463s 2023-06-23T17:59:43.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/peerstats.20230623: Permission denied 2023-06-23T17:59:43.684 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-06-23T17:59:43.685 INFO:teuthology.orchestra.run.smithi022.stderr:23 Jun 17:59:43 ntpd[22165]: can't open /var/log/ntpstats/loopstats.20230623: Permission denied 2023-06-23T17:59:43.739 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2023-06-23T17:59:43.740 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2023-06-23T17:59:43.740 INFO:teuthology.orchestra.run.smithi022.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:43.740 INFO:teuthology.orchestra.run.smithi022.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:43.740 INFO:teuthology.orchestra.run.smithi022.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:43.740 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-06-23T17:59:43.741 INFO:teuthology.run_tasks:Running task install... 2023-06-23T17:59:43.751 DEBUG:teuthology.task.install:project ceph 2023-06-23T17:59:43.751 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}} 2023-06-23T17:59:43.752 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'} 2023-06-23T17:59:43.752 INFO:teuthology.task.install:Using flavor: default 2023-06-23T17:59:43.758 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T17:59:43.758 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T17:59:43.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-key list | grep Ceph 2023-06-23T17:59:43.759 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-key list | grep Ceph 2023-06-23T17:59:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-06-23T17:59:43.865 INFO:teuthology.orchestra.run.smithi169.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-06-23T17:59:43.922 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-06-23T17:59:43.922 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph.com (release key) 2023-06-23T17:59:43.923 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-06-23T17:59:43.923 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-06-23T17:59:43.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T17:59:43.928 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-06-23T17:59:43.928 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph.com (release key) 2023-06-23T17:59:43.929 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-06-23T17:59:43.929 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-06-23T17:59:43.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T17:59:44.071 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default/ 2023-06-23T17:59:44.072 INFO:teuthology.task.install.deb:Package version is 18.0.0-4479-g8627af3e-1focal 2023-06-23T17:59:44.076 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default/ 2023-06-23T17:59:44.076 INFO:teuthology.task.install.deb:Package version is 18.0.0-4479-g8627af3e-1focal 2023-06-23T17:59:44.346 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T17:59:44.346 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-06-23T17:59:44.358 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T17:59:44.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-06-23T17:59:44.362 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get update 2023-06-23T17:59:44.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2023-06-23T17:59:44.520 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:59:44.521 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-06-23T17:59:44.526 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-06-23T17:59:44.543 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-06-23T17:59:44.631 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-06-23T17:59:44.636 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-06-23T17:59:44.702 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-06-23T17:59:44.785 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-06-23T17:59:44.819 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal InRelease 2023-06-23T17:59:44.839 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal InRelease 2023-06-23T17:59:45.016 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-06-23T17:59:45.039 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-06-23T17:59:45.040 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal Release.gpg 2023-06-23T17:59:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main i386 Packages [3,411 B] 2023-06-23T17:59:45.125 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal Release.gpg 2023-06-23T17:59:45.202 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-06-23T17:59:45.213 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-06-23T17:59:45.308 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 28.4 kB in 1s (37.4 kB/s) 2023-06-23T17:59:45.341 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main i386 Packages [3,411 B] 2023-06-23T17:59:45.433 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 28.4 kB in 1s (31.3 kB/s) 2023-06-23T17:59:46.337 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:59:46.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-4479-g8627af3e-1focal cephadm=18.0.0-4479-g8627af3e-1focal ceph-mds=18.0.0-4479-g8627af3e-1focal ceph-mgr=18.0.0-4479-g8627af3e-1focal ceph-common=18.0.0-4479-g8627af3e-1focal ceph-fuse=18.0.0-4479-g8627af3e-1focal ceph-test=18.0.0-4479-g8627af3e-1focal ceph-volume=18.0.0-4479-g8627af3e-1focal radosgw=18.0.0-4479-g8627af3e-1focal python3-rados=18.0.0-4479-g8627af3e-1focal python3-rgw=18.0.0-4479-g8627af3e-1focal python3-cephfs=18.0.0-4479-g8627af3e-1focal python3-rbd=18.0.0-4479-g8627af3e-1focal libcephfs2=18.0.0-4479-g8627af3e-1focal libcephfs-dev=18.0.0-4479-g8627af3e-1focal librados2=18.0.0-4479-g8627af3e-1focal librbd1=18.0.0-4479-g8627af3e-1focal rbd-fuse=18.0.0-4479-g8627af3e-1focal python3-cephfs=18.0.0-4479-g8627af3e-1focal cephfs-shell=18.0.0-4479-g8627af3e-1focal cephfs-top=18.0.0-4479-g8627af3e-1focal cephfs-mirror=18.0.0-4479-g8627af3e-1focal 2023-06-23T17:59:46.424 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:59:46.436 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T17:59:46.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-4479-g8627af3e-1focal cephadm=18.0.0-4479-g8627af3e-1focal ceph-mds=18.0.0-4479-g8627af3e-1focal ceph-mgr=18.0.0-4479-g8627af3e-1focal ceph-common=18.0.0-4479-g8627af3e-1focal ceph-fuse=18.0.0-4479-g8627af3e-1focal ceph-test=18.0.0-4479-g8627af3e-1focal ceph-volume=18.0.0-4479-g8627af3e-1focal radosgw=18.0.0-4479-g8627af3e-1focal python3-rados=18.0.0-4479-g8627af3e-1focal python3-rgw=18.0.0-4479-g8627af3e-1focal python3-cephfs=18.0.0-4479-g8627af3e-1focal python3-rbd=18.0.0-4479-g8627af3e-1focal libcephfs2=18.0.0-4479-g8627af3e-1focal libcephfs-dev=18.0.0-4479-g8627af3e-1focal librados2=18.0.0-4479-g8627af3e-1focal librbd1=18.0.0-4479-g8627af3e-1focal rbd-fuse=18.0.0-4479-g8627af3e-1focal python3-cephfs=18.0.0-4479-g8627af3e-1focal cephfs-shell=18.0.0-4479-g8627af3e-1focal cephfs-top=18.0.0-4479-g8627af3e-1focal cephfs-mirror=18.0.0-4479-g8627af3e-1focal 2023-06-23T17:59:46.523 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T17:59:46.619 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2023-06-23T17:59:46.619 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2023-06-23T17:59:46.717 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-06-23T17:59:46.717 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-06-23T17:59:46.791 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2023-06-23T17:59:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-06-23T17:59:46.792 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2023-06-23T17:59:46.792 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2023-06-23T17:59:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-06-23T17:59:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-06-23T17:59:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-06-23T17:59:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-06-23T17:59:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-06-23T17:59:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-06-23T17:59:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-06-23T17:59:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-06-23T17:59:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-06-23T17:59:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: qttranslations5-l10n socat xmlstarlet 2023-06-23T17:59:46.796 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2023-06-23T17:59:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-06-23T17:59:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-06-23T17:59:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-06-23T17:59:46.798 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2023-06-23T17:59:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: btrfs-tools 2023-06-23T17:59:46.888 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2023-06-23T17:59:46.889 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-06-23T17:59:46.889 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2023-06-23T17:59:46.889 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2023-06-23T17:59:46.889 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-06-23T17:59:46.890 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-06-23T17:59:46.890 INFO:teuthology.orchestra.run.smithi169.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-06-23T17:59:46.890 INFO:teuthology.orchestra.run.smithi169.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-06-23T17:59:46.891 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-06-23T17:59:46.892 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-06-23T17:59:46.893 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-06-23T17:59:46.893 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-06-23T17:59:46.893 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-06-23T17:59:46.893 INFO:teuthology.orchestra.run.smithi169.stdout: qttranslations5-l10n socat xmlstarlet 2023-06-23T17:59:46.893 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-06-23T17:59:46.894 INFO:teuthology.orchestra.run.smithi169.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-06-23T17:59:46.895 INFO:teuthology.orchestra.run.smithi169.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-06-23T17:59:46.895 INFO:teuthology.orchestra.run.smithi169.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-06-23T17:59:46.895 INFO:teuthology.orchestra.run.smithi169.stdout:Recommended packages: 2023-06-23T17:59:46.896 INFO:teuthology.orchestra.run.smithi169.stdout: btrfs-tools 2023-06-23T17:59:46.919 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2023-06-23T17:59:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-06-23T17:59:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-06-23T17:59:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-06-23T17:59:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-06-23T17:59:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-06-23T17:59:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-06-23T17:59:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-06-23T17:59:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-06-23T17:59:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-06-23T17:59:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-06-23T17:59:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-06-23T17:59:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-06-23T17:59:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-06-23T17:59:46.924 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2023-06-23T17:59:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 librbd1 2023-06-23T17:59:47.011 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2023-06-23T17:59:47.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-06-23T17:59:47.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-06-23T17:59:47.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-06-23T17:59:47.012 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-06-23T17:59:47.012 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-06-23T17:59:47.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-06-23T17:59:47.015 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-06-23T17:59:47.016 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2023-06-23T17:59:47.017 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 librbd1 2023-06-23T17:59:47.113 INFO:teuthology.orchestra.run.smithi022.stdout:2 upgraded, 116 newly installed, 0 to remove and 205 not upgraded. 2023-06-23T17:59:47.113 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 209 MB of archives. 2023-06-23T17:59:47.113 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 1,038 MB of additional disk space will be used. 2023-06-23T17:59:47.113 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-06-23T17:59:47.196 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-06-23T17:59:47.208 INFO:teuthology.orchestra.run.smithi169.stdout:2 upgraded, 116 newly installed, 0 to remove and 205 not upgraded. 2023-06-23T17:59:47.208 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 209 MB of archives. 2023-06-23T17:59:47.209 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 1,038 MB of additional disk space will be used. 2023-06-23T17:59:47.209 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-06-23T17:59:47.230 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-06-23T17:59:47.291 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-06-23T17:59:47.326 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-06-23T17:59:47.344 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-4479-g8627af3e-1focal [3,430 kB] 2023-06-23T17:59:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-06-23T17:59:47.427 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-4479-g8627af3e-1focal [3,430 kB] 2023-06-23T17:59:47.449 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-06-23T17:59:47.492 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-06-23T17:59:47.509 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-06-23T17:59:47.571 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-06-23T17:59:47.588 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-06-23T17:59:47.605 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-06-23T17:59:47.667 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-06-23T17:59:47.769 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-06-23T17:59:47.778 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-06-23T17:59:47.805 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-06-23T17:59:47.813 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-06-23T17:59:47.865 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-06-23T17:59:47.874 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-06-23T17:59:47.901 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-06-23T17:59:47.910 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-06-23T17:59:47.962 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-4479-g8627af3e-1focal [3,984 kB] 2023-06-23T17:59:47.978 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-06-23T17:59:48.073 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-06-23T17:59:48.132 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-4479-g8627af3e-1focal [725 kB] 2023-06-23T17:59:48.160 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-4479-g8627af3e-1focal [377 kB] 2023-06-23T17:59:48.170 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-4479-g8627af3e-1focal [22.5 kB] 2023-06-23T17:59:48.171 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-4479-g8627af3e-1focal [191 kB] 2023-06-23T17:59:48.176 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-4479-g8627af3e-1focal [48.4 kB] 2023-06-23T17:59:48.194 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-4479-g8627af3e-1focal [354 kB] 2023-06-23T17:59:48.210 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-4479-g8627af3e-1focal [6,769 kB] 2023-06-23T17:59:48.298 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-4479-g8627af3e-1focal [3,984 kB] 2023-06-23T17:59:48.336 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-06-23T17:59:48.355 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-06-23T17:59:48.431 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-06-23T17:59:48.450 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-06-23T17:59:48.532 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-06-23T17:59:48.542 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-06-23T17:59:48.576 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-06-23T17:59:48.582 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-06-23T17:59:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-06-23T17:59:48.587 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-06-23T17:59:48.597 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-06-23T17:59:48.598 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-06-23T17:59:48.615 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-4479-g8627af3e-1focal [104 kB] 2023-06-23T17:59:48.618 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-4479-g8627af3e-1focal [466 kB] 2023-06-23T17:59:48.624 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-4479-g8627af3e-1focal [25.5 MB] 2023-06-23T17:59:48.626 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-06-23T17:59:48.636 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-06-23T17:59:48.661 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-4479-g8627af3e-1focal [725 kB] 2023-06-23T17:59:48.670 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-06-23T17:59:48.676 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-06-23T17:59:48.678 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-06-23T17:59:48.681 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-06-23T17:59:48.691 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-06-23T17:59:48.692 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-06-23T17:59:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-4479-g8627af3e-1focal [377 kB] 2023-06-23T17:59:48.763 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-06-23T17:59:48.766 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-4479-g8627af3e-1focal [22.5 kB] 2023-06-23T17:59:48.771 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-4479-g8627af3e-1focal [191 kB] 2023-06-23T17:59:48.778 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-4479-g8627af3e-1focal [48.4 kB] 2023-06-23T17:59:48.781 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-4479-g8627af3e-1focal [354 kB] 2023-06-23T17:59:48.828 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-4479-g8627af3e-1focal [6,769 kB] 2023-06-23T17:59:48.857 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-06-23T17:59:49.080 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-06-23T17:59:49.104 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-06-23T17:59:49.173 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-06-23T17:59:49.173 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-06-23T17:59:49.195 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-06-23T17:59:49.236 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-06-23T17:59:49.244 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-06-23T17:59:49.264 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-06-23T17:59:49.290 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-4479-g8627af3e-1focal [104 kB] 2023-06-23T17:59:49.312 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-4479-g8627af3e-1focal [466 kB] 2023-06-23T17:59:49.325 INFO:teuthology.orchestra.run.smithi169.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-06-23T17:59:49.335 INFO:teuthology.orchestra.run.smithi169.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-06-23T17:59:49.348 INFO:teuthology.orchestra.run.smithi169.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-4479-g8627af3e-1focal [25.5 MB] 2023-06-23T17:59:49.364 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-06-23T17:59:49.364 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-06-23T17:59:49.368 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-06-23T17:59:49.372 INFO:teuthology.orchestra.run.smithi022.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-06-23T17:59:49.384 INFO:teuthology.orchestra.run.smithi022.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-06-23T17:59:49.455 INFO:teuthology.orchestra.run.smithi169.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-06-23T17:59:49.455 INFO:teuthology.orchestra.run.smithi169.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-06-23T17:59:49.460 INFO:teuthology.orchestra.run.smithi169.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-06-23T17:59:49.463 INFO:teuthology.orchestra.run.smithi169.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-06-23T17:59:49.475 INFO:teuthology.orchestra.run.smithi169.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-06-23T17:59:49.571 INFO:teuthology.orchestra.run.smithi022.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-06-23T17:59:49.660 INFO:teuthology.orchestra.run.smithi169.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-06-23T17:59:50.006 INFO:teuthology.orchestra.run.smithi022.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-06-23T17:59:50.018 INFO:teuthology.orchestra.run.smithi022.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-06-23T17:59:50.063 INFO:teuthology.orchestra.run.smithi022.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-06-23T17:59:50.095 INFO:teuthology.orchestra.run.smithi169.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-06-23T17:59:50.107 INFO:teuthology.orchestra.run.smithi169.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-06-23T17:59:50.107 INFO:teuthology.orchestra.run.smithi022.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-06-23T17:59:50.116 INFO:teuthology.orchestra.run.smithi022.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-06-23T17:59:50.121 INFO:teuthology.orchestra.run.smithi022.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-06-23T17:59:50.125 INFO:teuthology.orchestra.run.smithi022.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-06-23T17:59:50.151 INFO:teuthology.orchestra.run.smithi169.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-06-23T17:59:50.152 INFO:teuthology.orchestra.run.smithi022.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-06-23T17:59:50.172 INFO:teuthology.orchestra.run.smithi022.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-06-23T17:59:50.195 INFO:teuthology.orchestra.run.smithi169.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-06-23T17:59:50.204 INFO:teuthology.orchestra.run.smithi169.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-06-23T17:59:50.210 INFO:teuthology.orchestra.run.smithi169.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-06-23T17:59:50.213 INFO:teuthology.orchestra.run.smithi169.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-06-23T17:59:50.240 INFO:teuthology.orchestra.run.smithi169.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-06-23T17:59:50.254 INFO:teuthology.orchestra.run.smithi022.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-06-23T17:59:50.256 INFO:teuthology.orchestra.run.smithi022.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-4479-g8627af3e-1focal [11.0 MB] 2023-06-23T17:59:50.261 INFO:teuthology.orchestra.run.smithi169.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-06-23T17:59:50.272 INFO:teuthology.orchestra.run.smithi169.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-06-23T17:59:50.419 INFO:teuthology.orchestra.run.smithi022.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-06-23T17:59:50.437 INFO:teuthology.orchestra.run.smithi169.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-06-23T17:59:50.609 INFO:teuthology.orchestra.run.smithi022.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-06-23T17:59:50.627 INFO:teuthology.orchestra.run.smithi169.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-06-23T17:59:51.010 INFO:teuthology.orchestra.run.smithi022.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-4479-g8627af3e-1focal [223 kB] 2023-06-23T17:59:51.012 INFO:teuthology.orchestra.run.smithi022.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-4479-g8627af3e-1focal [133 kB] 2023-06-23T17:59:51.030 INFO:teuthology.orchestra.run.smithi022.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-4479-g8627af3e-1focal [1,550 kB] 2023-06-23T17:59:51.077 INFO:teuthology.orchestra.run.smithi169.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-4479-g8627af3e-1focal [11.0 MB] 2023-06-23T17:59:51.118 INFO:teuthology.orchestra.run.smithi022.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-4479-g8627af3e-1focal [9,393 kB] 2023-06-23T17:59:51.378 INFO:teuthology.orchestra.run.smithi022.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-06-23T17:59:51.394 INFO:teuthology.orchestra.run.smithi169.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-06-23T17:59:51.456 INFO:teuthology.orchestra.run.smithi022.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-06-23T17:59:51.458 INFO:teuthology.orchestra.run.smithi022.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-06-23T17:59:51.467 INFO:teuthology.orchestra.run.smithi169.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-06-23T17:59:51.469 INFO:teuthology.orchestra.run.smithi169.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-06-23T17:59:51.496 INFO:teuthology.orchestra.run.smithi022.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-06-23T17:59:51.497 INFO:teuthology.orchestra.run.smithi022.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-06-23T17:59:51.508 INFO:teuthology.orchestra.run.smithi169.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-06-23T17:59:51.509 INFO:teuthology.orchestra.run.smithi169.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-06-23T17:59:51.509 INFO:teuthology.orchestra.run.smithi169.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-06-23T17:59:51.509 INFO:teuthology.orchestra.run.smithi169.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-06-23T17:59:51.510 INFO:teuthology.orchestra.run.smithi169.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-06-23T17:59:51.510 INFO:teuthology.orchestra.run.smithi169.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-06-23T17:59:51.524 INFO:teuthology.orchestra.run.smithi022.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-06-23T17:59:51.525 INFO:teuthology.orchestra.run.smithi022.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-06-23T17:59:51.526 INFO:teuthology.orchestra.run.smithi022.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-06-23T17:59:51.527 INFO:teuthology.orchestra.run.smithi022.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-06-23T17:59:51.674 INFO:teuthology.orchestra.run.smithi169.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-06-23T17:59:51.692 INFO:teuthology.orchestra.run.smithi022.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-06-23T17:59:51.766 INFO:teuthology.orchestra.run.smithi169.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-4479-g8627af3e-1focal [223 kB] 2023-06-23T17:59:51.770 INFO:teuthology.orchestra.run.smithi169.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-4479-g8627af3e-1focal [133 kB] 2023-06-23T17:59:51.773 INFO:teuthology.orchestra.run.smithi169.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-4479-g8627af3e-1focal [1,550 kB] 2023-06-23T17:59:51.784 INFO:teuthology.orchestra.run.smithi022.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-4479-g8627af3e-1focal [31.3 MB] 2023-06-23T17:59:51.832 INFO:teuthology.orchestra.run.smithi169.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-4479-g8627af3e-1focal [9,393 kB] 2023-06-23T17:59:51.990 INFO:teuthology.orchestra.run.smithi169.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-06-23T17:59:52.003 INFO:teuthology.orchestra.run.smithi169.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-06-23T17:59:52.010 INFO:teuthology.orchestra.run.smithi022.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-06-23T17:59:52.023 INFO:teuthology.orchestra.run.smithi022.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-06-23T17:59:52.030 INFO:teuthology.orchestra.run.smithi169.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-06-23T17:59:52.050 INFO:teuthology.orchestra.run.smithi022.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-06-23T17:59:52.185 INFO:teuthology.orchestra.run.smithi169.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-06-23T17:59:52.205 INFO:teuthology.orchestra.run.smithi022.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-06-23T17:59:52.230 INFO:teuthology.orchestra.run.smithi169.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-06-23T17:59:52.248 INFO:teuthology.orchestra.run.smithi022.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-06-23T17:59:52.258 INFO:teuthology.orchestra.run.smithi169.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-06-23T17:59:52.267 INFO:teuthology.orchestra.run.smithi169.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-06-23T17:59:52.276 INFO:teuthology.orchestra.run.smithi022.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-06-23T17:59:52.285 INFO:teuthology.orchestra.run.smithi022.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-06-23T17:59:52.309 INFO:teuthology.orchestra.run.smithi169.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-06-23T17:59:52.328 INFO:teuthology.orchestra.run.smithi022.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-06-23T17:59:52.340 INFO:teuthology.orchestra.run.smithi169.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-06-23T17:59:52.342 INFO:teuthology.orchestra.run.smithi169.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-06-23T17:59:52.358 INFO:teuthology.orchestra.run.smithi022.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-06-23T17:59:52.361 INFO:teuthology.orchestra.run.smithi022.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-06-23T17:59:52.407 INFO:teuthology.orchestra.run.smithi169.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-4479-g8627af3e-1focal [31.3 MB] 2023-06-23T17:59:52.624 INFO:teuthology.orchestra.run.smithi169.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-06-23T17:59:52.642 INFO:teuthology.orchestra.run.smithi022.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-06-23T17:59:53.095 INFO:teuthology.orchestra.run.smithi169.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-06-23T17:59:53.112 INFO:teuthology.orchestra.run.smithi022.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-06-23T17:59:53.150 INFO:teuthology.orchestra.run.smithi169.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-06-23T17:59:53.167 INFO:teuthology.orchestra.run.smithi022.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-06-23T17:59:53.497 INFO:teuthology.orchestra.run.smithi169.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-06-23T17:59:53.499 INFO:teuthology.orchestra.run.smithi169.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-06-23T17:59:53.499 INFO:teuthology.orchestra.run.smithi169.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-06-23T17:59:53.500 INFO:teuthology.orchestra.run.smithi169.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-06-23T17:59:53.501 INFO:teuthology.orchestra.run.smithi169.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-06-23T17:59:53.503 INFO:teuthology.orchestra.run.smithi169.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-06-23T17:59:53.515 INFO:teuthology.orchestra.run.smithi022.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-06-23T17:59:53.516 INFO:teuthology.orchestra.run.smithi022.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-06-23T17:59:53.517 INFO:teuthology.orchestra.run.smithi022.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-06-23T17:59:53.518 INFO:teuthology.orchestra.run.smithi022.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-06-23T17:59:53.519 INFO:teuthology.orchestra.run.smithi022.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-06-23T17:59:53.520 INFO:teuthology.orchestra.run.smithi022.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-06-23T17:59:53.543 INFO:teuthology.orchestra.run.smithi169.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-06-23T17:59:53.543 INFO:teuthology.orchestra.run.smithi169.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-06-23T17:59:53.560 INFO:teuthology.orchestra.run.smithi022.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-06-23T17:59:53.561 INFO:teuthology.orchestra.run.smithi022.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-06-23T17:59:53.708 INFO:teuthology.orchestra.run.smithi169.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-06-23T17:59:53.726 INFO:teuthology.orchestra.run.smithi022.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-06-23T17:59:53.875 INFO:teuthology.orchestra.run.smithi169.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-06-23T17:59:53.893 INFO:teuthology.orchestra.run.smithi022.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-06-23T17:59:54.088 INFO:teuthology.orchestra.run.smithi169.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-06-23T17:59:54.136 INFO:teuthology.orchestra.run.smithi169.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-06-23T17:59:54.148 INFO:teuthology.orchestra.run.smithi169.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-06-23T17:59:54.160 INFO:teuthology.orchestra.run.smithi022.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-06-23T17:59:54.176 INFO:teuthology.orchestra.run.smithi022.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-06-23T17:59:54.177 INFO:teuthology.orchestra.run.smithi022.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-06-23T17:59:54.211 INFO:teuthology.orchestra.run.smithi169.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-06-23T17:59:54.226 INFO:teuthology.orchestra.run.smithi169.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-06-23T17:59:54.227 INFO:teuthology.orchestra.run.smithi022.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-4479-g8627af3e-1focal [3,804 B] 2023-06-23T17:59:54.260 INFO:teuthology.orchestra.run.smithi022.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-06-23T17:59:54.261 INFO:teuthology.orchestra.run.smithi022.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-4479-g8627af3e-1focal [832 kB] 2023-06-23T17:59:54.262 INFO:teuthology.orchestra.run.smithi022.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-06-23T17:59:54.276 INFO:teuthology.orchestra.run.smithi022.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-4479-g8627af3e-1focal [2,295 kB] 2023-06-23T17:59:54.355 INFO:teuthology.orchestra.run.smithi022.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-4479-g8627af3e-1focal [213 kB] 2023-06-23T17:59:54.372 INFO:teuthology.orchestra.run.smithi022.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-4479-g8627af3e-1focal [123 kB] 2023-06-23T17:59:54.375 INFO:teuthology.orchestra.run.smithi022.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-4479-g8627af3e-1focal [1,775 kB] 2023-06-23T17:59:54.467 INFO:teuthology.orchestra.run.smithi169.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-4479-g8627af3e-1focal [3,804 B] 2023-06-23T17:59:54.468 INFO:teuthology.orchestra.run.smithi169.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-4479-g8627af3e-1focal [832 kB] 2023-06-23T17:59:54.482 INFO:teuthology.orchestra.run.smithi169.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-4479-g8627af3e-1focal [2,295 kB] 2023-06-23T17:59:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-4479-g8627af3e-1focal [7,885 kB] 2023-06-23T17:59:54.523 INFO:teuthology.orchestra.run.smithi169.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-4479-g8627af3e-1focal [213 kB] 2023-06-23T17:59:54.556 INFO:teuthology.orchestra.run.smithi169.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-4479-g8627af3e-1focal [123 kB] 2023-06-23T17:59:54.560 INFO:teuthology.orchestra.run.smithi169.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-4479-g8627af3e-1focal [1,775 kB] 2023-06-23T17:59:54.661 INFO:teuthology.orchestra.run.smithi169.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-4479-g8627af3e-1focal [7,885 kB] 2023-06-23T17:59:55.112 INFO:teuthology.orchestra.run.smithi169.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-4479-g8627af3e-1focal [3,928 B] 2023-06-23T17:59:55.113 INFO:teuthology.orchestra.run.smithi169.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-4479-g8627af3e-1focal [50.8 MB] 2023-06-23T17:59:55.117 INFO:teuthology.orchestra.run.smithi022.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-4479-g8627af3e-1focal [3,928 B] 2023-06-23T17:59:55.118 INFO:teuthology.orchestra.run.smithi022.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-4479-g8627af3e-1focal [50.8 MB] 2023-06-23T17:59:58.033 INFO:teuthology.orchestra.run.smithi169.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-4479-g8627af3e-1focal [220 kB] 2023-06-23T17:59:58.037 INFO:teuthology.orchestra.run.smithi169.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-4479-g8627af3e-1focal [256 kB] 2023-06-23T17:59:58.042 INFO:teuthology.orchestra.run.smithi169.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-4479-g8627af3e-1focal [23.3 kB] 2023-06-23T17:59:58.043 INFO:teuthology.orchestra.run.smithi169.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-4479-g8627af3e-1focal [12.9 kB] 2023-06-23T17:59:58.080 INFO:teuthology.orchestra.run.smithi169.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-4479-g8627af3e-1focal [28.0 kB] 2023-06-23T17:59:58.081 INFO:teuthology.orchestra.run.smithi169.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-4479-g8627af3e-1focal [13.7 MB] 2023-06-23T17:59:58.628 INFO:teuthology.orchestra.run.smithi169.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-4479-g8627af3e-1focal [97.2 kB] 2023-06-23T17:59:59.188 INFO:teuthology.orchestra.run.smithi022.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-4479-g8627af3e-1focal [220 kB] 2023-06-23T17:59:59.192 INFO:teuthology.orchestra.run.smithi022.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-4479-g8627af3e-1focal [256 kB] 2023-06-23T17:59:59.198 INFO:teuthology.orchestra.run.smithi022.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-4479-g8627af3e-1focal [23.3 kB] 2023-06-23T17:59:59.198 INFO:teuthology.orchestra.run.smithi022.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-4479-g8627af3e-1focal [12.9 kB] 2023-06-23T17:59:59.199 INFO:teuthology.orchestra.run.smithi022.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-4479-g8627af3e-1focal [28.0 kB] 2023-06-23T17:59:59.200 INFO:teuthology.orchestra.run.smithi022.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-4479-g8627af3e-1focal [13.7 MB] 2023-06-23T17:59:59.333 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 209 MB in 12s (17.9 MB/s) 2023-06-23T17:59:59.478 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyperclip. 2023-06-23T17:59:59.530 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-06-23T17:59:59.535 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-06-23T17:59:59.570 INFO:teuthology.orchestra.run.smithi022.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-4479-g8627af3e-1focal [97.2 kB] 2023-06-23T17:59:59.576 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-06-23T17:59:59.929 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-wcwidth. 2023-06-23T17:59:59.945 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-06-23T17:59:59.987 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-06-23T18:00:00.278 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 209 MB in 13s (16.5 MB/s) 2023-06-23T18:00:00.282 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cmd2. 2023-06-23T18:00:00.298 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-06-23T18:00:00.332 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-06-23T18:00:00.414 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyperclip. 2023-06-23T18:00:00.471 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-06-23T18:00:00.475 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-06-23T18:00:00.511 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-06-23T18:00:00.702 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-06-23T18:00:00.720 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-06-23T18:00:00.797 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-wcwidth. 2023-06-23T18:00:00.817 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-06-23T18:00:00.827 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-06-23T18:00:00.855 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-06-23T18:00:01.166 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cmd2. 2023-06-23T18:00:01.183 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-06-23T18:00:01.225 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-06-23T18:00:01.228 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-06-23T18:00:01.244 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-06-23T18:00:01.286 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-06-23T18:00:01.657 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-06-23T18:00:01.673 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-06-23T18:00:01.694 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-06-23T18:00:01.713 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-06-23T18:00:01.748 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-06-23T18:00:01.794 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-06-23T18:00:02.165 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-06-23T18:00:02.168 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-06-23T18:00:02.182 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-06-23T18:00:02.185 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:02.223 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-06-23T18:00:02.228 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:02.551 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-06-23T18:00:02.571 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-06-23T18:00:02.609 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-06-23T18:00:02.763 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-06-23T18:00:02.780 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:02.822 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:03.059 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-06-23T18:00:03.079 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:03.127 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:03.402 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-06-23T18:00:03.420 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:03.451 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:03.654 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-06-23T18:00:03.670 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:03.712 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:03.889 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-06-23T18:00:03.905 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-06-23T18:00:03.939 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-06-23T18:00:04.282 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-06-23T18:00:04.302 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-06-23T18:00:04.340 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:00:04.351 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../010-librbd1_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:04.502 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librbd1 (18.0.0-4479-g8627af3e-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-06-23T18:00:04.845 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-06-23T18:00:04.864 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-06-23T18:00:04.903 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-06-23T18:00:05.192 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../011-librados2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:05.273 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../010-librbd1_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:05.332 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librados2 (18.0.0-4479-g8627af3e-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-06-23T18:00:05.398 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librbd1 (18.0.0-4479-g8627af3e-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-06-23T18:00:06.086 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs2. 2023-06-23T18:00:06.094 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../011-librados2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:06.104 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:06.144 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:06.221 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librados2 (18.0.0-4479-g8627af3e-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-06-23T18:00:06.505 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rados. 2023-06-23T18:00:06.524 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../013-python3-rados_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:06.563 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rados (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:06.925 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs2. 2023-06-23T18:00:06.933 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-06-23T18:00:06.942 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:06.950 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:06.984 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:06.992 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-argparse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:07.269 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cephfs. 2023-06-23T18:00:07.285 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:07.327 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cephfs (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:07.419 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rados. 2023-06-23T18:00:07.438 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../013-python3-rados_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:07.477 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rados (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:07.696 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-common. 2023-06-23T18:00:07.715 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:07.755 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:07.822 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-06-23T18:00:07.841 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:07.880 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-argparse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:08.100 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-prettytable. 2023-06-23T18:00:08.119 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-06-23T18:00:08.157 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cephfs. 2023-06-23T18:00:08.158 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-06-23T18:00:08.177 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:08.215 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cephfs (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:08.444 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rbd. 2023-06-23T18:00:08.460 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:08.494 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rbd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:08.543 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-common. 2023-06-23T18:00:08.559 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:08.601 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:08.822 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-06-23T18:00:08.851 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-06-23T18:00:08.893 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-06-23T18:00:08.921 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-prettytable. 2023-06-23T18:00:08.941 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-06-23T18:00:08.979 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-06-23T18:00:09.240 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rbd. 2023-06-23T18:00:09.251 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-06-23T18:00:09.257 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:09.268 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-06-23T18:00:09.298 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rbd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:09.308 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-06-23T18:00:09.626 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-06-23T18:00:09.645 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-06-23T18:00:09.652 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-06-23T18:00:09.671 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-06-23T18:00:09.684 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-06-23T18:00:09.711 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-06-23T18:00:10.022 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librgw2. 2023-06-23T18:00:10.040 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../022-librgw2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:10.045 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-06-23T18:00:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-06-23T18:00:10.103 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-06-23T18:00:10.113 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librgw2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:10.348 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-06-23T18:00:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-06-23T18:00:10.398 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-06-23T18:00:10.734 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librgw2. 2023-06-23T18:00:10.750 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../022-librgw2_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:10.792 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librgw2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:11.008 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rgw. 2023-06-23T18:00:11.025 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:11.108 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:11.479 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liboath0. 2023-06-23T18:00:11.497 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-06-23T18:00:11.537 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-06-23T18:00:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rgw. 2023-06-23T18:00:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:11.874 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libradosstriper1. 2023-06-23T18:00:11.892 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:11.932 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libradosstriper1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:12.064 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liboath0. 2023-06-23T18:00:12.079 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-06-23T18:00:12.113 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-06-23T18:00:12.375 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libradosstriper1. 2023-06-23T18:00:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:12.434 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libradosstriper1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:12.475 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-common. 2023-06-23T18:00:12.493 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../026-ceph-common_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:12.533 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:12.929 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-common. 2023-06-23T18:00:12.946 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../026-ceph-common_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:12.987 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:15.073 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-base. 2023-06-23T18:00:15.089 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../027-ceph-base_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:15.140 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-base (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:15.498 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-base. 2023-06-23T18:00:15.516 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../027-ceph-base_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-base (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:16.628 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cherrypy3. 2023-06-23T18:00:16.647 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-06-23T18:00:16.686 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-06-23T18:00:16.961 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cherrypy3. 2023-06-23T18:00:16.980 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-06-23T18:00:17.019 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-06-23T18:00:17.056 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-natsort. 2023-06-23T18:00:17.073 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-06-23T18:00:17.114 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-06-23T18:00:17.381 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-natsort. 2023-06-23T18:00:17.397 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-06-23T18:00:17.439 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-06-23T18:00:17.478 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-logutils. 2023-06-23T18:00:17.496 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-06-23T18:00:17.536 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-06-23T18:00:17.759 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-logutils. 2023-06-23T18:00:17.775 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-06-23T18:00:17.817 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-06-23T18:00:17.864 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-markupsafe. 2023-06-23T18:00:17.881 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-06-23T18:00:17.922 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-06-23T18:00:18.078 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-markupsafe. 2023-06-23T18:00:18.096 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-06-23T18:00:18.127 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-06-23T18:00:18.258 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-mako. 2023-06-23T18:00:18.276 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-06-23T18:00:18.358 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-06-23T18:00:18.406 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-mako. 2023-06-23T18:00:18.422 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-06-23T18:00:18.464 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-06-23T18:00:18.759 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplegeneric. 2023-06-23T18:00:18.775 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-06-23T18:00:18.817 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-06-23T18:00:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-singledispatch. 2023-06-23T18:00:19.172 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-06-23T18:00:19.211 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-06-23T18:00:19.470 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplegeneric. 2023-06-23T18:00:19.487 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-06-23T18:00:19.528 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-06-23T18:00:19.839 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-singledispatch. 2023-06-23T18:00:19.859 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-06-23T18:00:19.898 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-06-23T18:00:20.114 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webob. 2023-06-23T18:00:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-06-23T18:00:20.172 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-06-23T18:00:20.255 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webob. 2023-06-23T18:00:20.275 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-06-23T18:00:20.317 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-06-23T18:00:20.525 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-soupsieve. 2023-06-23T18:00:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-06-23T18:00:20.583 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-06-23T18:00:20.680 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-soupsieve. 2023-06-23T18:00:20.696 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-06-23T18:00:20.738 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-06-23T18:00:20.844 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-bs4. 2023-06-23T18:00:20.864 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-06-23T18:00:20.894 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-06-23T18:00:21.065 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-bs4. 2023-06-23T18:00:21.082 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-06-23T18:00:21.123 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-06-23T18:00:21.155 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-waitress. 2023-06-23T18:00:21.175 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-06-23T18:00:21.208 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-06-23T18:00:21.468 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-tempita. 2023-06-23T18:00:21.487 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-06-23T18:00:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-06-23T18:00:21.535 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-waitress. 2023-06-23T18:00:21.553 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-06-23T18:00:21.597 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-06-23T18:00:21.787 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-paste. 2023-06-23T18:00:21.807 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-06-23T18:00:21.845 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-06-23T18:00:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-tempita. 2023-06-23T18:00:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-06-23T18:00:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-06-23T18:00:22.182 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-06-23T18:00:22.198 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-06-23T18:00:22.240 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-06-23T18:00:22.383 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-paste. 2023-06-23T18:00:22.403 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-06-23T18:00:22.441 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-06-23T18:00:22.534 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastedeploy. 2023-06-23T18:00:22.552 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-06-23T18:00:22.592 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-06-23T18:00:22.802 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-06-23T18:00:22.820 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-06-23T18:00:22.860 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-06-23T18:00:22.895 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webtest. 2023-06-23T18:00:22.913 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-06-23T18:00:22.953 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-06-23T18:00:23.171 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastedeploy. 2023-06-23T18:00:23.191 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-06-23T18:00:23.255 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-06-23T18:00:23.264 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pecan. 2023-06-23T18:00:23.281 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-06-23T18:00:23.322 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-06-23T18:00:23.599 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webtest. 2023-06-23T18:00:23.616 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-06-23T18:00:23.657 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-06-23T18:00:23.711 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-werkzeug. 2023-06-23T18:00:23.731 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-06-23T18:00:23.769 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-06-23T18:00:24.011 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pecan. 2023-06-23T18:00:24.029 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-06-23T18:00:24.064 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-06-23T18:00:24.069 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-06-23T18:00:24.080 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:24.122 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-modules-core (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:24.449 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-werkzeug. 2023-06-23T18:00:24.465 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-06-23T18:00:24.515 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-06-23T18:00:24.543 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-06-23T18:00:24.559 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:24.601 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:24.810 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-06-23T18:00:24.830 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:24.868 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-modules-core (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:24.904 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cryptography. 2023-06-23T18:00:24.924 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-06-23T18:00:24.962 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-06-23T18:00:25.256 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-06-23T18:00:25.273 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:25.335 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:25.367 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-openssl. 2023-06-23T18:00:25.382 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-06-23T18:00:25.424 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-06-23T18:00:25.701 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cryptography. 2023-06-23T18:00:25.717 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-06-23T18:00:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr. 2023-06-23T18:00:25.759 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-06-23T18:00:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:25.802 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:26.154 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-openssl. 2023-06-23T18:00:26.175 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-06-23T18:00:26.238 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-06-23T18:00:26.288 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mon. 2023-06-23T18:00:26.305 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:26.346 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mon (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:26.607 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr. 2023-06-23T18:00:26.624 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:26.665 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:27.177 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mon. 2023-06-23T18:00:27.194 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:27.235 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mon (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:27.600 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-osd. 2023-06-23T18:00:27.618 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:27.658 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-osd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:28.505 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-osd. 2023-06-23T18:00:28.524 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:28.563 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-osd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:30.728 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph. 2023-06-23T18:00:30.745 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../053-ceph_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:30.786 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:31.105 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-fuse. 2023-06-23T18:00:31.122 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:31.163 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:31.566 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mds. 2023-06-23T18:00:31.584 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:31.616 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mds (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:31.725 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph. 2023-06-23T18:00:31.741 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../053-ceph_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:32.144 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-fuse. 2023-06-23T18:00:32.161 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephadm. 2023-06-23T18:00:32.161 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:32.180 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../056-cephadm_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:32.202 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:32.219 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:32.538 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-asyncssh. 2023-06-23T18:00:32.554 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-06-23T18:00:32.604 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-06-23T18:00:32.672 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mds. 2023-06-23T18:00:32.689 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:32.730 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mds (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:32.999 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jinja2. 2023-06-23T18:00:33.020 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-06-23T18:00:33.057 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-06-23T18:00:33.316 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephadm. 2023-06-23T18:00:33.333 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../056-cephadm_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-06-23T18:00:33.374 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:33.388 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:33.426 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-asyncssh. 2023-06-23T18:00:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-06-23T18:00:33.780 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-bcrypt. 2023-06-23T18:00:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-06-23T18:00:33.799 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-06-23T18:00:33.837 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-06-23T18:00:34.140 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jwt. 2023-06-23T18:00:34.157 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-06-23T18:00:34.190 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-06-23T18:00:34.197 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jinja2. 2023-06-23T18:00:34.216 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-06-23T18:00:34.272 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-06-23T18:00:34.493 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-repoze.lru. 2023-06-23T18:00:34.512 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2023-06-23T18:00:34.552 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-06-23T18:00:34.616 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-06-23T18:00:34.633 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:34.674 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:34.804 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjs-underscore. 2023-06-23T18:00:34.820 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-06-23T18:00:34.862 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-06-23T18:00:35.114 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-06-23T18:00:35.130 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-06-23T18:00:35.172 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-06-23T18:00:35.460 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-routes. 2023-06-23T18:00:35.461 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-bcrypt. 2023-06-23T18:00:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2023-06-23T18:00:35.480 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-06-23T18:00:35.519 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-06-23T18:00:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-06-23T18:00:35.864 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jwt. 2023-06-23T18:00:35.881 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-06-23T18:00:35.922 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-06-23T18:00:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-06-23T18:00:36.163 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:36.204 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-dashboard (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:36.275 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-repoze.lru. 2023-06-23T18:00:36.291 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2023-06-23T18:00:36.358 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-06-23T18:00:36.644 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-underscore. 2023-06-23T18:00:36.661 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-06-23T18:00:36.700 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-decorator. 2023-06-23T18:00:36.702 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-06-23T18:00:36.716 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-06-23T18:00:36.766 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-06-23T18:00:36.988 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-06-23T18:00:37.005 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-06-23T18:00:37.038 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-06-23T18:00:37.052 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-06-23T18:00:37.069 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-06-23T18:00:37.110 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-06-23T18:00:37.307 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-routes. 2023-06-23T18:00:37.327 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2023-06-23T18:00:37.365 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-06-23T18:00:37.421 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-scipy. 2023-06-23T18:00:37.438 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-06-23T18:00:37.479 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-06-23T18:00:37.677 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-06-23T18:00:37.693 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:37.727 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-dashboard (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:38.239 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-decorator. 2023-06-23T18:00:38.259 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-06-23T18:00:38.297 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-06-23T18:00:38.633 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-06-23T18:00:38.650 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-06-23T18:00:38.716 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-06-23T18:00:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-06-23T18:00:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-06-23T18:00:38.780 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-06-23T18:00:39.078 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-scipy. 2023-06-23T18:00:39.093 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-06-23T18:00:39.135 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-06-23T18:00:39.350 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-joblib. 2023-06-23T18:00:39.366 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2023-06-23T18:00:39.408 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-06-23T18:00:39.770 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn. 2023-06-23T18:00:39.786 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-06-23T18:00:39.828 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-06-23T18:00:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-06-23T18:00:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:40.544 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-06-23T18:00:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:40.561 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-06-23T18:00:40.602 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-06-23T18:00:41.172 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-joblib. 2023-06-23T18:00:41.192 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2023-06-23T18:00:41.230 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-06-23T18:00:41.625 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn. 2023-06-23T18:00:41.641 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-06-23T18:00:41.647 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cachetools. 2023-06-23T18:00:41.663 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2023-06-23T18:00:41.705 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-06-23T18:00:41.726 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-06-23T18:00:42.008 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1. 2023-06-23T18:00:42.024 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-06-23T18:00:42.066 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-06-23T18:00:42.417 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-06-23T18:00:42.436 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:42.453 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-06-23T18:00:42.470 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-06-23T18:00:42.474 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:42.510 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-06-23T18:00:42.822 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rsa. 2023-06-23T18:00:42.840 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2023-06-23T18:00:42.880 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rsa (4.0-3) ... 2023-06-23T18:00:43.191 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-google-auth. 2023-06-23T18:00:43.212 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2023-06-23T18:00:43.249 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-06-23T18:00:43.511 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-blinker. 2023-06-23T18:00:43.527 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-06-23T18:00:43.561 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-06-23T18:00:43.678 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cachetools. 2023-06-23T18:00:43.696 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2023-06-23T18:00:43.736 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-06-23T18:00:43.855 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-oauthlib. 2023-06-23T18:00:43.872 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-06-23T18:00:43.913 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-06-23T18:00:44.055 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1. 2023-06-23T18:00:44.076 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-06-23T18:00:44.113 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-06-23T18:00:44.294 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-06-23T18:00:44.311 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-06-23T18:00:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-06-23T18:00:44.508 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-06-23T18:00:44.525 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-06-23T18:00:44.566 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-06-23T18:00:44.647 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-websocket. 2023-06-23T18:00:44.667 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-06-23T18:00:44.705 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-06-23T18:00:44.903 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rsa. 2023-06-23T18:00:44.921 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2023-06-23T18:00:44.961 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rsa (4.0-3) ... 2023-06-23T18:00:45.026 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-kubernetes. 2023-06-23T18:00:45.042 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-06-23T18:00:45.084 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-06-23T18:00:45.305 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-google-auth. 2023-06-23T18:00:45.322 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2023-06-23T18:00:45.380 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-06-23T18:00:45.614 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-06-23T18:00:45.631 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:45.672 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:45.725 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-blinker. 2023-06-23T18:00:45.744 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-06-23T18:00:45.783 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-06-23T18:00:45.992 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libonig5:amd64. 2023-06-23T18:00:46.008 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2023-06-23T18:00:46.050 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-06-23T18:00:46.111 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-oauthlib. 2023-06-23T18:00:46.127 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-06-23T18:00:46.169 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-06-23T18:00:46.411 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjq1:amd64. 2023-06-23T18:00:46.432 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:00:46.469 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:00:46.625 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-06-23T18:00:46.641 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-06-23T18:00:46.683 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-06-23T18:00:46.722 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package jq. 2023-06-23T18:00:46.740 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:00:46.780 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:00:47.020 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-websocket. 2023-06-23T18:00:47.036 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-06-23T18:00:47.058 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package socat. 2023-06-23T18:00:47.077 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2023-06-23T18:00:47.078 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-06-23T18:00:47.116 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking socat (1.7.3.3-2) ... 2023-06-23T18:00:47.406 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-kubernetes. 2023-06-23T18:00:47.419 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package xmlstarlet. 2023-06-23T18:00:47.424 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-06-23T18:00:47.436 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-06-23T18:00:47.477 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-06-23T18:00:47.489 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-06-23T18:00:47.798 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-test. 2023-06-23T18:00:47.817 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../090-ceph-test_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:47.856 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-test (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:48.071 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-06-23T18:00:48.089 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:48.128 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:48.472 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libonig5:amd64. 2023-06-23T18:00:48.489 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2023-06-23T18:00:48.531 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-06-23T18:00:48.959 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjq1:amd64. 2023-06-23T18:00:48.977 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:00:49.017 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:00:49.312 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package jq. 2023-06-23T18:00:49.329 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:00:49.370 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:00:49.656 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package socat. 2023-06-23T18:00:49.673 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2023-06-23T18:00:49.714 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking socat (1.7.3.3-2) ... 2023-06-23T18:00:50.067 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package xmlstarlet. 2023-06-23T18:00:50.083 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-06-23T18:00:50.158 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-06-23T18:00:50.530 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-test. 2023-06-23T18:00:50.548 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../090-ceph-test_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:50.588 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-test (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:52.344 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-volume. 2023-06-23T18:00:52.361 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../091-ceph-volume_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:52.402 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-volume (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:52.773 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-mirror. 2023-06-23T18:00:52.793 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../092-cephfs-mirror_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:52.831 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-mirror (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:53.201 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-colorama. 2023-06-23T18:00:53.218 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2023-06-23T18:00:53.259 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-06-23T18:00:53.520 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-shell. 2023-06-23T18:00:53.540 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../094-cephfs-shell_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:53.578 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-shell (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:53.822 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-top. 2023-06-23T18:00:53.841 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../095-cephfs-top_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:53.880 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-top (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:54.133 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package fonts-mathjax. 2023-06-23T18:00:54.150 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-06-23T18:00:54.191 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:00:54.770 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs-dev. 2023-06-23T18:00:54.789 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../097-libcephfs-dev_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:54.820 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs-dev (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:55.125 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-volume. 2023-06-23T18:00:55.144 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../091-ceph-volume_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:55.196 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-volume (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:55.206 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package nvme-cli. 2023-06-23T18:00:55.223 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-06-23T18:00:55.282 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-06-23T18:00:55.586 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-06-23T18:00:55.603 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-06-23T18:00:55.604 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-mirror. 2023-06-23T18:00:55.621 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../092-cephfs-mirror_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:55.644 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-06-23T18:00:55.663 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-mirror (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:56.005 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjs-mathjax. 2023-06-23T18:00:56.024 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-colorama. 2023-06-23T18:00:56.024 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-06-23T18:00:56.041 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2023-06-23T18:00:56.063 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:00:56.115 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-06-23T18:00:56.410 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-shell. 2023-06-23T18:00:56.426 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../094-cephfs-shell_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:56.468 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-shell (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:56.737 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-top. 2023-06-23T18:00:56.755 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../095-cephfs-top_18.0.0-4479-g8627af3e-1focal_all.deb ... 2023-06-23T18:00:56.795 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-top (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:57.073 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package fonts-mathjax. 2023-06-23T18:00:57.090 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-06-23T18:00:57.131 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:00:57.676 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs-dev. 2023-06-23T18:00:57.696 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../097-libcephfs-dev_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:00:57.707 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-natsort-doc. 2023-06-23T18:00:57.727 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2023-06-23T18:00:57.734 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs-dev (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:00:57.798 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-06-23T18:00:58.126 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-atomicwrites. 2023-06-23T18:00:58.143 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../102-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-06-23T18:00:58.162 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package nvme-cli. 2023-06-23T18:00:58.179 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-06-23T18:00:58.184 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-06-23T18:00:58.263 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-06-23T18:00:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-attr. 2023-06-23T18:00:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../103-python3-attr_19.3.0-2_all.deb ... 2023-06-23T18:00:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-06-23T18:00:58.590 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-06-23T18:00:58.606 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-06-23T18:00:58.639 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-06-23T18:00:58.848 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webencodings. 2023-06-23T18:00:58.869 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../104-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-06-23T18:00:58.914 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-06-23T18:00:59.034 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-mathjax. 2023-06-23T18:00:59.050 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-06-23T18:00:59.125 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:00:59.202 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-html5lib. 2023-06-23T18:00:59.221 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../105-python3-html5lib_1.0.1-2_all.deb ... 2023-06-23T18:00:59.259 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-06-23T18:00:59.638 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-06-23T18:00:59.657 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../106-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-06-23T18:00:59.696 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-06-23T18:01:00.157 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-packaging. 2023-06-23T18:01:00.175 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../107-python3-packaging_20.3-1_all.deb ... 2023-06-23T18:01:00.215 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-packaging (20.3-1) ... 2023-06-23T18:01:00.518 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastescript. 2023-06-23T18:01:00.536 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../108-python3-pastescript_2.0.2-4_all.deb ... 2023-06-23T18:01:00.576 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-06-23T18:01:00.816 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-natsort-doc. 2023-06-23T18:01:00.835 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2023-06-23T18:01:00.874 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-06-23T18:01:00.930 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pluggy. 2023-06-23T18:01:00.947 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../109-python3-pluggy_0.13.0-2_all.deb ... 2023-06-23T18:01:00.988 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-06-23T18:01:01.237 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-atomicwrites. 2023-06-23T18:01:01.254 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../102-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-06-23T18:01:01.316 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-psutil. 2023-06-23T18:01:01.320 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-06-23T18:01:01.333 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../110-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-06-23T18:01:01.374 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-06-23T18:01:01.656 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-attr. 2023-06-23T18:01:01.674 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../103-python3-attr_19.3.0-2_all.deb ... 2023-06-23T18:01:01.710 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-py. 2023-06-23T18:01:01.714 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-06-23T18:01:01.729 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../111-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-06-23T18:01:01.768 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-06-23T18:01:02.075 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webencodings. 2023-06-23T18:01:02.093 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../104-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-06-23T18:01:02.098 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyinotify. 2023-06-23T18:01:02.116 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-06-23T18:01:02.133 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-06-23T18:01:02.156 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-06-23T18:01:02.493 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pytest. 2023-06-23T18:01:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-html5lib. 2023-06-23T18:01:02.513 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../113-python3-pytest_4.6.9-1_all.deb ... 2023-06-23T18:01:02.514 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../105-python3-html5lib_1.0.1-2_all.deb ... 2023-06-23T18:01:02.550 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-06-23T18:01:02.552 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-06-23T18:01:02.896 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplejson. 2023-06-23T18:01:02.914 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-06-23T18:01:02.917 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-06-23T18:01:02.933 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../106-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-06-23T18:01:02.953 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-06-23T18:01:02.980 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-06-23T18:01:03.256 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-06-23T18:01:03.274 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-06-23T18:01:03.314 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-06-23T18:01:03.475 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-packaging. 2023-06-23T18:01:03.496 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../107-python3-packaging_20.3-1_all.deb ... 2023-06-23T18:01:03.525 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-packaging (20.3-1) ... 2023-06-23T18:01:03.794 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastescript. 2023-06-23T18:01:03.813 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../108-python3-pastescript_2.0.2-4_all.deb ... 2023-06-23T18:01:03.844 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-06-23T18:01:04.010 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package radosgw. 2023-06-23T18:01:04.029 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../116-radosgw_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:01:04.076 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking radosgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:04.181 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pluggy. 2023-06-23T18:01:04.198 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../109-python3-pluggy_0.13.0-2_all.deb ... 2023-06-23T18:01:04.264 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-06-23T18:01:04.584 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-psutil. 2023-06-23T18:01:04.604 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../110-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-06-23T18:01:04.642 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-06-23T18:01:05.003 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-py. 2023-06-23T18:01:05.022 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../111-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-06-23T18:01:05.061 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-06-23T18:01:05.456 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyinotify. 2023-06-23T18:01:05.474 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-06-23T18:01:05.515 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-06-23T18:01:05.571 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package rbd-fuse. 2023-06-23T18:01:05.590 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:01:05.620 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking rbd-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:05.961 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-attr (19.3.0-2) ... 2023-06-23T18:01:06.326 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pytest. 2023-06-23T18:01:06.342 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-06-23T18:01:06.342 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../113-python3-pytest_4.6.9-1_all.deb ... 2023-06-23T18:01:06.384 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-06-23T18:01:06.754 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplejson. 2023-06-23T18:01:06.770 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-06-23T18:01:06.828 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-06-23T18:01:06.835 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-06-23T18:01:06.962 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-06-23T18:01:07.140 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-06-23T18:01:07.158 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-06-23T18:01:07.188 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:07.198 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-06-23T18:01:07.824 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user cephadm....done 2023-06-23T18:01:07.936 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package radosgw. 2023-06-23T18:01:07.944 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:01:07.954 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../116-radosgw_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:01:08.019 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking radosgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:08.070 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-06-23T18:01:08.431 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-06-23T18:01:08.724 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:01:08.850 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-06-23T18:01:08.976 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-06-23T18:01:09.302 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-06-23T18:01:09.514 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package rbd-fuse. 2023-06-23T18:01:09.531 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-4479-g8627af3e-1focal_amd64.deb ... 2023-06-23T18:01:09.605 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking rbd-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:09.704 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-06-23T18:01:09.950 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-attr (19.3.0-2) ... 2023-06-23T18:01:10.017 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-06-23T18:01:10.243 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-06-23T18:01:10.294 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-06-23T18:01:10.369 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-06-23T18:01:10.529 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-06-23T18:01:10.561 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-06-23T18:01:10.756 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:10.837 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-06-23T18:01:11.122 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-argparse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:11.409 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user cephadm....done 2023-06-23T18:01:11.431 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-06-23T18:01:11.554 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:01:11.557 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-06-23T18:01:11.708 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-06-23T18:01:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-06-23T18:01:12.043 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-06-23T18:01:12.134 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-06-23T18:01:12.336 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-06-23T18:01:12.418 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-06-23T18:01:12.462 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-06-23T18:01:12.527 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-06-23T18:01:12.613 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-06-23T18:01:12.895 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-06-23T18:01:12.940 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-06-23T18:01:13.181 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-06-23T18:01:13.334 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-06-23T18:01:13.473 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-06-23T18:01:13.618 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-06-23T18:01:13.808 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-06-23T18:01:13.903 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-06-23T18:01:14.084 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-06-23T18:01:14.187 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-06-23T18:01:14.410 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-06-23T18:01:14.472 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-06-23T18:01:14.703 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-packaging (20.3-1) ... 2023-06-23T18:01:14.790 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-argparse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:15.007 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-06-23T18:01:15.074 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-06-23T18:01:15.133 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-06-23T18:01:15.225 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-06-23T18:01:15.259 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-06-23T18:01:15.519 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-06-23T18:01:15.544 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-06-23T18:01:15.819 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-06-23T18:01:15.946 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-06-23T18:01:16.121 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-06-23T18:01:16.240 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-06-23T18:01:16.249 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-06-23T18:01:16.598 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-06-23T18:01:16.601 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-06-23T18:01:16.902 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-06-23T18:01:16.952 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up socat (1.7.3.3-2) ... 2023-06-23T18:01:17.078 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:17.220 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-06-23T18:01:17.396 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:17.530 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-06-23T18:01:17.622 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-06-23T18:01:17.915 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-06-23T18:01:17.922 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-06-23T18:01:18.024 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-06-23T18:01:18.150 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-06-23T18:01:18.265 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-06-23T18:01:18.434 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-06-23T18:01:18.560 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-packaging (20.3-1) ... 2023-06-23T18:01:18.760 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:18.853 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-06-23T18:01:18.887 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-06-23T18:01:18.978 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-06-23T18:01:19.012 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-06-23T18:01:19.138 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-06-23T18:01:19.331 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-06-23T18:01:19.414 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-06-23T18:01:19.640 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-06-23T18:01:19.807 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-06-23T18:01:19.933 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-06-23T18:01:20.059 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-06-23T18:01:20.117 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-06-23T18:01:20.335 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-06-23T18:01:20.468 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-06-23T18:01:20.644 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-06-23T18:01:20.870 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up socat (1.7.3.3-2) ... 2023-06-23T18:01:20.945 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-06-23T18:01:20.996 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:21.314 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:21.344 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-06-23T18:01:21.482 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-06-23T18:01:21.623 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-06-23T18:01:21.740 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-06-23T18:01:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-06-23T18:01:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-06-23T18:01:22.050 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-06-23T18:01:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-06-23T18:01:22.386 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-06-23T18:01:22.444 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-06-23T18:01:22.611 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-06-23T18:01:22.712 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:22.737 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-06-23T18:01:22.838 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-06-23T18:01:22.997 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-06-23T18:01:23.074 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-06-23T18:01:23.324 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-06-23T18:01:23.359 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-06-23T18:01:23.608 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-06-23T18:01:23.902 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-06-23T18:01:24.028 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-06-23T18:01:24.404 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-06-23T18:01:24.715 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-06-23T18:01:24.864 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:01:24.993 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-06-23T18:01:25.025 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-06-23T18:01:25.361 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-06-23T18:01:25.405 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-06-23T18:01:25.669 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-06-23T18:01:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-06-23T18:01:25.795 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-06-23T18:01:25.980 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:26.106 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-06-23T18:01:26.172 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-06-23T18:01:26.441 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-06-23T18:01:26.534 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-06-23T18:01:26.660 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-06-23T18:01:26.778 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-06-23T18:01:26.837 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-06-23T18:01:27.146 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-06-23T18:01:27.179 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-06-23T18:01:27.472 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-06-23T18:01:27.489 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-06-23T18:01:28.561 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-06-23T18:01:28.686 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-06-23T18:01:28.837 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:01:28.955 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:01:28.963 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rsa (4.0-3) ... 2023-06-23T18:01:29.089 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-06-23T18:01:29.273 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-06-23T18:01:29.390 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-06-23T18:01:29.524 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-06-23T18:01:29.675 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-06-23T18:01:29.833 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-06-23T18:01:29.976 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-06-23T18:01:30.126 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-06-23T18:01:30.244 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-06-23T18:01:30.311 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-06-23T18:01:30.637 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-06-23T18:01:30.671 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librados2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:30.883 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-06-23T18:01:30.955 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-06-23T18:01:31.009 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librgw2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:31.135 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libsqlite3-mod-ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:31.261 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:31.350 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-06-23T18:01:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-routes (2.4.1-2) ... 2023-06-23T18:01:31.693 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-06-23T18:01:31.705 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libradosstriper1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:31.856 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-06-23T18:01:32.149 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librbd1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-modules-core (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:32.410 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:32.641 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-06-23T18:01:32.641 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-06-23T18:01:32.765 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-06-23T18:01:32.892 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-06-23T18:01:33.051 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-06-23T18:01:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs-dev (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:33.177 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rsa (4.0-3) ... 2023-06-23T18:01:33.237 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rados (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:33.388 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-top (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:33.470 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-06-23T18:01:33.514 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-06-23T18:01:33.629 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-06-23T18:01:33.963 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-06-23T18:01:34.256 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-06-23T18:01:34.459 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rbd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:34.574 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-06-23T18:01:34.585 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up rbd-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:34.720 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:34.845 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cephfs (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:34.918 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librados2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:34.996 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-shell (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:35.114 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:35.163 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-06-23T18:01:35.289 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librgw2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:35.440 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libsqlite3-mod-ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:35.575 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs2 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:Adding group ceph....done 2023-06-23T18:01:35.726 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-routes (2.4.1-2) ... 2023-06-23T18:01:36.020 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libradosstriper1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user ceph....done 2023-06-23T18:01:36.145 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-06-23T18:01:36.194 INFO:teuthology.orchestra.run.smithi022.stdout:Setting system user ceph properties....done 2023-06-23T18:01:36.217 INFO:teuthology.orchestra.run.smithi022.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-06-23T18:01:36.367 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-06-23T18:01:36.480 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librbd1 (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:36.640 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-modules-core (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:36.688 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-06-23T18:01:36.766 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:37.027 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-06-23T18:01:37.027 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-06-23T18:01:37.140 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-mirror (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:37.432 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-06-23T18:01:37.432 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-06-23T18:01:37.501 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs-dev (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:37.627 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rados (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:37.778 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-top (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:37.904 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-06-23T18:01:38.235 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-test (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:38.360 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up radosgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-06-23T18:01:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-06-23T18:01:38.833 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rbd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:38.993 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up rbd-fuse (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.119 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.269 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cephfs (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.304 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-base (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.396 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-shell (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.555 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-common (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:39.710 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-06-23T18:01:40.158 INFO:teuthology.orchestra.run.smithi169.stdout:Adding group ceph....done 2023-06-23T18:01:40.164 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mds (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-06-23T18:01:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-06-23T18:01:40.551 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user ceph....done 2023-06-23T18:01:40.769 INFO:teuthology.orchestra.run.smithi169.stdout:Setting system user ceph properties....done 2023-06-23T18:01:40.789 INFO:teuthology.orchestra.run.smithi169.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-06-23T18:01:40.849 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-06-23T18:01:41.079 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-06-23T18:01:41.080 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-06-23T18:01:41.247 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-06-23T18:01:41.517 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-osd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:41.698 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-mirror (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:41.925 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-06-23T18:01:41.925 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-06-23T18:01:42.021 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-06-23T18:01:42.021 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-06-23T18:01:42.383 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-test (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:42.469 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-k8sevents (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:42.568 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up radosgw (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:42.594 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:42.754 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mon (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:42.858 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-06-23T18:01:42.859 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-06-23T18:01:43.004 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-06-23T18:01:43.005 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-06-23T18:01:43.456 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-base (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:43.599 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:43.725 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-dashboard (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:43.879 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-volume (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:43.918 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-06-23T18:01:44.005 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-06-23T18:01:44.174 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-06-23T18:01:44.355 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mds (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:44.483 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-06-23T18:01:44.577 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-06-23T18:01:44.577 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-06-23T18:01:45.026 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:45.281 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-06-23T18:01:45.282 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-06-23T18:01:45.728 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-osd (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:46.222 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-06-23T18:01:46.222 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-06-23T18:01:46.679 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-k8sevents (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:46.805 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:46.981 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mon (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:47.208 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-06-23T18:01:47.208 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-06-23T18:01:47.527 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-06-23T18:01:47.676 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-cephadm (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:47.843 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:47.953 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-dashboard (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:48.104 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-volume (18.0.0-4479-g8627af3e-1focal) ... 2023-06-23T18:01:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-06-23T18:01:48.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-06-23T18:01:48.189 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T18:01:48.214 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-06-23T18:01:48.390 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2023-06-23T18:01:48.391 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2023-06-23T18:01:48.399 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-06-23T18:01:48.581 INFO:teuthology.orchestra.run.smithi022.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-06-23T18:01:48.581 INFO:teuthology.orchestra.run.smithi022.stdout:flex is already the newest version (2.6.4-6.2). 2023-06-23T18:01:48.581 INFO:teuthology.orchestra.run.smithi022.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-06-23T18:01:48.582 INFO:teuthology.orchestra.run.smithi022.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-06-23T18:01:48.582 INFO:teuthology.orchestra.run.smithi022.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-06-23T18:01:48.582 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2023-06-23T18:01:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-06-23T18:01:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2023-06-23T18:01:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2023-06-23T18:01:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-06-23T18:01:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 libuuid1 2023-06-23T18:01:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: modemmanager network-manager-pptp postgresql-12 postgresql-client-12 2023-06-23T18:01:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-06-23T18:01:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: wpasupplicant 2023-06-23T18:01:48.587 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2023-06-23T18:01:48.587 INFO:teuthology.orchestra.run.smithi022.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-06-23T18:01:48.587 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-06-23T18:01:48.588 INFO:teuthology.orchestra.run.smithi022.stdout: libengine-pkcs11-openssl 2023-06-23T18:01:48.637 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2023-06-23T18:01:48.638 INFO:teuthology.orchestra.run.smithi022.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-06-23T18:01:48.638 INFO:teuthology.orchestra.run.smithi022.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-06-23T18:01:48.639 INFO:teuthology.orchestra.run.smithi022.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-06-23T18:01:48.639 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-06-23T18:01:48.639 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-06-23T18:01:48.640 INFO:teuthology.orchestra.run.smithi022.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-06-23T18:01:48.640 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2023-06-23T18:01:48.640 INFO:teuthology.orchestra.run.smithi022.stdout: libssl1.1 libuuid1 util-linux uuid-dev 2023-06-23T18:01:48.693 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-06-23T18:01:48.695 INFO:teuthology.orchestra.run.smithi022.stdout:4 upgraded, 31 newly installed, 0 to remove and 201 not upgraded. 2023-06-23T18:01:48.695 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 25.1 MB of archives. 2023-06-23T18:01:48.695 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-06-23T18:01:48.696 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-06-23T18:01:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-06-23T18:01:48.826 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-06-23T18:01:48.827 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-06-23T18:01:48.846 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-06-23T18:01:48.846 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-06-23T18:01:48.847 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-06-23T18:01:48.848 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-06-23T18:01:48.849 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-06-23T18:01:48.850 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-06-23T18:01:48.850 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-06-23T18:01:48.886 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-06-23T18:01:48.905 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-06-23T18:01:48.965 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-06-23T18:01:48.966 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-06-23T18:01:48.990 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.15-0ubuntu0.20.04.1 [116 kB] 2023-06-23T18:01:48.995 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-06-23T18:01:49.011 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-06-23T18:01:49.011 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-06-23T18:01:49.030 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-06-23T18:01:49.030 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-06-23T18:01:49.038 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-06-23T18:01:49.083 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-06-23T18:01:49.230 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-06-23T18:01:49.237 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-06-23T18:01:49.238 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-06-23T18:01:49.239 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-06-23T18:01:49.239 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.15-0ubuntu0.20.04.1 [1,052 kB] 2023-06-23T18:01:49.251 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-06-23T18:01:49.252 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.15-0ubuntu0.20.04.1 [13.5 MB] 2023-06-23T18:01:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-06-23T18:01:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-06-23T18:01:49.442 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-06-23T18:01:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-06-23T18:01:49.517 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-06-23T18:01:49.954 INFO:teuthology.orchestra.run.smithi022.stdout:Preconfiguring packages ... 2023-06-23T18:01:50.126 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 25.1 MB in 1s (28.2 MB/s) 2023-06-23T18:01:50.429 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:01:50.434 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:01:50.589 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:01:51.572 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-06-23T18:01:51.690 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-06-23T18:01:52.218 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-06-23T18:01:52.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-06-23T18:01:52.296 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-06-23T18:01:52.492 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-06-23T18:01:52.493 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-06-23T18:01:52.676 INFO:teuthology.orchestra.run.smithi169.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-06-23T18:01:52.676 INFO:teuthology.orchestra.run.smithi169.stdout:flex is already the newest version (2.6.4-6.2). 2023-06-23T18:01:52.677 INFO:teuthology.orchestra.run.smithi169.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-06-23T18:01:52.677 INFO:teuthology.orchestra.run.smithi169.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-06-23T18:01:52.677 INFO:teuthology.orchestra.run.smithi169.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-06-23T18:01:52.677 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2023-06-23T18:01:52.678 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-06-23T18:01:52.678 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2023-06-23T18:01:52.678 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2023-06-23T18:01:52.678 INFO:teuthology.orchestra.run.smithi169.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-06-23T18:01:52.679 INFO:teuthology.orchestra.run.smithi169.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 libuuid1 2023-06-23T18:01:52.679 INFO:teuthology.orchestra.run.smithi169.stdout: modemmanager network-manager-pptp postgresql-12 postgresql-client-12 2023-06-23T18:01:52.679 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-06-23T18:01:52.679 INFO:teuthology.orchestra.run.smithi169.stdout: wpasupplicant 2023-06-23T18:01:52.679 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2023-06-23T18:01:52.680 INFO:teuthology.orchestra.run.smithi169.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-06-23T18:01:52.680 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-06-23T18:01:52.680 INFO:teuthology.orchestra.run.smithi169.stdout: libengine-pkcs11-openssl 2023-06-23T18:01:52.735 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2023-06-23T18:01:52.736 INFO:teuthology.orchestra.run.smithi169.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-06-23T18:01:52.736 INFO:teuthology.orchestra.run.smithi169.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-06-23T18:01:52.737 INFO:teuthology.orchestra.run.smithi169.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-06-23T18:01:52.737 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-06-23T18:01:52.737 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-06-23T18:01:52.737 INFO:teuthology.orchestra.run.smithi169.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-06-23T18:01:52.738 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2023-06-23T18:01:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: libssl1.1 libuuid1 util-linux uuid-dev 2023-06-23T18:01:52.922 INFO:teuthology.orchestra.run.smithi169.stdout:4 upgraded, 31 newly installed, 0 to remove and 201 not upgraded. 2023-06-23T18:01:52.923 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 25.1 MB of archives. 2023-06-23T18:01:52.923 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-06-23T18:01:52.923 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-06-23T18:01:53.137 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:01:53.142 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:01:53.266 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:01:53.546 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-06-23T18:01:53.547 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-06-23T18:01:53.548 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-06-23T18:01:53.647 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-06-23T18:01:53.649 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-06-23T18:01:53.656 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-06-23T18:01:53.662 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-06-23T18:01:53.662 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-06-23T18:01:53.665 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-06-23T18:01:53.668 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-06-23T18:01:53.699 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:01:53.840 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-06-23T18:01:53.864 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:01:53.924 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-06-23T18:01:54.157 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-06-23T18:01:54.216 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-06-23T18:01:54.221 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-06-23T18:01:54.341 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.15-0ubuntu0.20.04.1 [116 kB] 2023-06-23T18:01:54.361 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-06-23T18:01:54.439 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-06-23T18:01:54.440 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-06-23T18:01:54.531 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-06-23T18:01:54.531 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-06-23T18:01:54.561 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-06-23T18:01:54.636 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:01:54.641 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-06-23T18:01:54.758 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-06-23T18:01:54.763 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-06-23T18:01:55.301 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-06-23T18:01:55.319 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-06-23T18:01:55.359 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-06-23T18:01:55.454 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-06-23T18:01:55.463 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-06-23T18:01:55.465 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-06-23T18:01:55.466 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-06-23T18:01:55.466 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.15-0ubuntu0.20.04.1 [1,052 kB] 2023-06-23T18:01:55.499 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-06-23T18:01:55.505 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.15-0ubuntu0.20.04.1 [13.5 MB] 2023-06-23T18:01:55.704 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package dump. 2023-06-23T18:01:55.721 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-06-23T18:01:55.762 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking dump (0.4b46-6) ... 2023-06-23T18:01:55.955 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-06-23T18:01:55.956 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-06-23T18:01:55.956 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-06-23T18:01:56.040 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package indent. 2023-06-23T18:01:56.057 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-06-23T18:01:56.098 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking indent (2.2.12-1) ... 2023-06-23T18:01:56.120 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-06-23T18:01:56.314 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-06-23T18:01:56.393 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-06-23T18:01:56.411 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-06-23T18:01:56.451 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-06-23T18:01:56.796 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-06-23T18:01:56.804 INFO:teuthology.orchestra.run.smithi169.stdout:Preconfiguring packages ... 2023-06-23T18:01:56.815 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-06-23T18:01:56.854 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-06-23T18:01:57.032 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 25.1 MB in 4s (6,841 kB/s) 2023-06-23T18:01:57.132 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-06-23T18:01:57.149 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-06-23T18:01:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-06-23T18:01:57.286 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:01:57.291 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:01:57.467 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:01:57.527 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-06-23T18:01:57.544 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:01:57.601 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:01:57.862 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-proxy. 2023-06-23T18:01:57.882 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:01:57.920 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:01:58.474 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-06-23T18:01:58.494 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:01:58.665 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-06-23T18:01:58.732 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:01:59.921 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libndp0:amd64. 2023-06-23T18:01:59.941 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-06-23T18:01:59.978 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-06-23T18:02:00.118 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:02:00.123 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:02:00.283 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:02:00.315 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libnm0:amd64. 2023-06-23T18:02:00.332 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-06-23T18:02:00.373 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:00.673 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:02:00.709 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpq5:amd64. 2023-06-23T18:02:00.727 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../12-libpq5_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:00.759 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:00.873 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:02:01.087 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-06-23T18:02:01.106 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:01.170 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:01.233 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-06-23T18:02:01.497 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-proxy. 2023-06-23T18:02:01.515 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:01.556 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:01.808 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-06-23T18:02:01.819 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152867 files and directories currently installed.) 2023-06-23T18:02:01.823 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-06-23T18:02:01.825 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-06-23T18:02:01.866 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:01.990 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-06-23T18:02:02.403 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-06-23T18:02:02.423 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-06-23T18:02:02.461 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-06-23T18:02:02.612 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-06-23T18:02:02.632 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-06-23T18:02:02.804 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-06-23T18:02:02.856 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package modemmanager. 2023-06-23T18:02:02.873 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:02.925 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:03.450 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package dump. 2023-06-23T18:02:03.469 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-06-23T18:02:03.508 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking dump (0.4b46-6) ... 2023-06-23T18:02:03.520 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package wpasupplicant. 2023-06-23T18:02:03.537 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-06-23T18:02:03.578 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-06-23T18:02:03.837 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package indent. 2023-06-23T18:02:03.857 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-06-23T18:02:03.895 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking indent (2.2.12-1) ... 2023-06-23T18:02:04.108 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager. 2023-06-23T18:02:04.129 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-06-23T18:02:04.191 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:04.214 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-06-23T18:02:04.233 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-06-23T18:02:04.273 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-06-23T18:02:04.668 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-06-23T18:02:04.686 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-06-23T18:02:04.726 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-06-23T18:02:05.029 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-06-23T18:02:05.049 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-06-23T18:02:05.059 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ppp. 2023-06-23T18:02:05.078 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-06-23T18:02:05.087 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-06-23T18:02:05.365 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-06-23T18:02:05.407 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-06-23T18:02:05.426 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:05.457 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:05.705 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package pptp-linux. 2023-06-23T18:02:05.724 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-06-23T18:02:05.763 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-06-23T18:02:05.777 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-proxy. 2023-06-23T18:02:05.796 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:05.868 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:06.100 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager-pptp. 2023-06-23T18:02:06.117 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-06-23T18:02:06.176 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-06-23T18:02:06.221 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-06-23T18:02:06.238 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:06.279 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:06.461 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-common. 2023-06-23T18:02:06.482 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-06-23T18:02:06.519 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-06-23T18:02:06.624 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libndp0:amd64. 2023-06-23T18:02:06.641 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-06-23T18:02:06.682 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-06-23T18:02:06.823 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-12. 2023-06-23T18:02:06.843 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../24-postgresql-client-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:06.881 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:07.061 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libnm0:amd64. 2023-06-23T18:02:07.078 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-06-23T18:02:07.119 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:07.495 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-common. 2023-06-23T18:02:07.512 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-06-23T18:02:07.530 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpq5:amd64. 2023-06-23T18:02:07.549 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../12-libpq5_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:07.577 INFO:teuthology.orchestra.run.smithi022.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-06-23T18:02:07.604 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-06-23T18:02:07.608 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:07.975 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-06-23T18:02:07.993 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:08.025 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:08.076 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-12. 2023-06-23T18:02:08.095 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../26-postgresql-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:08.137 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:09.022 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-proxy. 2023-06-23T18:02:09.041 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:09.079 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:09.365 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-06-23T18:02:09.383 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-06-23T18:02:09.423 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:09.728 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql. 2023-06-23T18:02:09.750 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:09.786 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-06-23T18:02:09.969 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-06-23T18:02:09.986 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-06-23T18:02:10.027 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-06-23T18:02:10.030 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client. 2023-06-23T18:02:10.048 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:10.088 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-06-23T18:02:10.456 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package modemmanager. 2023-06-23T18:02:10.476 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-06-23T18:02:10.524 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:10.549 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-contrib. 2023-06-23T18:02:10.572 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:10.927 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-06-23T18:02:11.193 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-06-23T18:02:11.204 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package wpasupplicant. 2023-06-23T18:02:11.210 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-06-23T18:02:11.222 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-06-23T18:02:11.251 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-06-23T18:02:11.304 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-06-23T18:02:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch. 2023-06-23T18:02:11.634 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-06-23T18:02:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-06-23T18:02:11.926 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager. 2023-06-23T18:02:11.943 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-06-23T18:02:11.988 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-06-23T18:02:12.008 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:12.235 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:12.528 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:12.649 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ppp. 2023-06-23T18:02:12.653 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-06-23T18:02:12.666 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-06-23T18:02:12.780 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-06-23T18:02:12.905 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-06-23T18:02:12.923 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-06-23T18:02:13.301 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-06-23T18:02:13.302 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-06-23T18:02:13.314 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package pptp-linux. 2023-06-23T18:02:13.333 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-06-23T18:02:13.379 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-06-23T18:02:13.766 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager-pptp. 2023-06-23T18:02:13.783 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-06-23T18:02:13.792 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:13.850 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-06-23T18:02:14.395 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-common. 2023-06-23T18:02:14.412 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-06-23T18:02:14.453 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-06-23T18:02:14.688 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-06-23T18:02:14.789 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-06-23T18:02:14.833 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-12. 2023-06-23T18:02:14.852 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../24-postgresql-client-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:14.891 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:14.906 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-06-23T18:02:15.015 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-06-23T18:02:15.554 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-common. 2023-06-23T18:02:15.571 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-06-23T18:02:15.636 INFO:teuthology.orchestra.run.smithi169.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-06-23T18:02:15.664 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-06-23T18:02:16.042 INFO:teuthology.orchestra.run.smithi022.stdout:Adding user postgres to group ssl-cert 2023-06-23T18:02:16.203 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-12. 2023-06-23T18:02:16.219 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../26-postgresql-12_12.15-0ubuntu0.20.04.1_amd64.deb ... 2023-06-23T18:02:16.264 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:16.318 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:16.318 INFO:teuthology.orchestra.run.smithi022.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-06-23T18:02:16.576 INFO:teuthology.orchestra.run.smithi022.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-06-23T18:02:16.576 INFO:teuthology.orchestra.run.smithi022.stdout:Removing obsolete dictionary files: 2023-06-23T18:02:16.907 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-06-23T18:02:17.659 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:17.785 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-06-23T18:02:17.911 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-06-23T18:02:17.989 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql. 2023-06-23T18:02:18.007 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:18.037 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:18.047 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-06-23T18:02:18.162 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-06-23T18:02:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client. 2023-06-23T18:02:18.344 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:18.391 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-06-23T18:02:18.669 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-contrib. 2023-06-23T18:02:18.686 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-06-23T18:02:18.752 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-06-23T18:02:19.029 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-06-23T18:02:19.049 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-06-23T18:02:19.087 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-06-23T18:02:19.128 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-06-23T18:02:19.368 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up dump (0.4b46-6) ... 2023-06-23T18:02:19.465 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-06-23T18:02:19.491 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch. 2023-06-23T18:02:19.512 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-06-23T18:02:19.528 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:19.559 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-06-23T18:02:19.654 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-06-23T18:02:19.805 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up indent (2.2.12-1) ... 2023-06-23T18:02:19.894 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-06-23T18:02:19.930 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-06-23T18:02:20.107 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-06-23T18:02:20.138 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:20.225 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:20.448 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpq5:amd64 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:Creating new PostgreSQL cluster 12/main ... 2023-06-23T18:02:20.607 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-06-23T18:02:20.617 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-06-23T18:02:20.677 INFO:teuthology.orchestra.run.smithi022.stdout:The files belonging to this database system will be owned by user "postgres". 2023-06-23T18:02:20.677 INFO:teuthology.orchestra.run.smithi022.stdout:This user must also own the server process. 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout:The database cluster will be initialized with locale "en_US". 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout:The default text search configuration will be set to "english". 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:20.678 INFO:teuthology.orchestra.run.smithi022.stdout:Data page checksums are disabled. 2023-06-23T18:02:20.679 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:20.679 INFO:teuthology.orchestra.run.smithi022.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-06-23T18:02:20.679 INFO:teuthology.orchestra.run.smithi022.stdout:creating subdirectories ... ok 2023-06-23T18:02:20.680 INFO:teuthology.orchestra.run.smithi022.stdout:selecting dynamic shared memory implementation ... posix 2023-06-23T18:02:20.698 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default max_connections ... 100 2023-06-23T18:02:20.715 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default shared_buffers ... 128MB 2023-06-23T18:02:20.722 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default time zone ... Etc/UTC 2023-06-23T18:02:20.722 INFO:teuthology.orchestra.run.smithi022.stdout:creating configuration files ... ok 2023-06-23T18:02:20.727 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-06-23T18:02:20.834 INFO:teuthology.orchestra.run.smithi022.stdout:running bootstrap script ... ok 2023-06-23T18:02:20.886 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-06-23T18:02:21.321 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-06-23T18:02:21.321 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-06-23T18:02:21.481 INFO:teuthology.orchestra.run.smithi022.stdout:performing post-bootstrap initialization ... ok 2023-06-23T18:02:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:22.426 INFO:teuthology.orchestra.run.smithi022.stdout:syncing data to disk ... ok 2023-06-23T18:02:22.426 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:22.426 INFO:teuthology.orchestra.run.smithi022.stdout:Success. You can now start the database server using: 2023-06-23T18:02:22.426 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:22.427 INFO:teuthology.orchestra.run.smithi022.stdout: pg_ctlcluster 12 main start 2023-06-23T18:02:22.427 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:22.691 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-06-23T18:02:22.772 INFO:teuthology.orchestra.run.smithi022.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-06-23T18:02:22.773 INFO:teuthology.orchestra.run.smithi022.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-06-23T18:02:22.786 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-06-23T18:02:22.859 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-06-23T18:02:22.946 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-06-23T18:02:23.072 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-06-23T18:02:24.156 INFO:teuthology.orchestra.run.smithi169.stdout:Adding user postgres to group ssl-cert 2023-06-23T18:02:24.406 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:24.406 INFO:teuthology.orchestra.run.smithi169.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-06-23T18:02:24.858 INFO:teuthology.orchestra.run.smithi169.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-06-23T18:02:24.859 INFO:teuthology.orchestra.run.smithi169.stdout:Removing obsolete dictionary files: 2023-06-23T18:02:25.191 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-06-23T18:02:25.848 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:25.965 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-06-23T18:02:25.991 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-06-23T18:02:26.100 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-06-23T18:02:26.158 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-06-23T18:02:26.234 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-06-23T18:02:26.284 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:26.344 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:26.492 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-06-23T18:02:27.009 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:27.010 INFO:teuthology.orchestra.run.smithi022.stdout:The following network interfaces were found in /etc/network/interfaces 2023-06-23T18:02:27.010 INFO:teuthology.orchestra.run.smithi022.stdout:which means they are currently configured by ifupdown: 2023-06-23T18:02:27.011 INFO:teuthology.orchestra.run.smithi022.stdout:- ens1f0 2023-06-23T18:02:27.011 INFO:teuthology.orchestra.run.smithi022.stdout:If you want to manage those interfaces with NetworkManager instead 2023-06-23T18:02:27.011 INFO:teuthology.orchestra.run.smithi022.stdout:remove their configuration from /etc/network/interfaces. 2023-06-23T18:02:27.012 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:27.176 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-06-23T18:02:27.496 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-06-23T18:02:27.505 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-06-23T18:02:27.734 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up dump (0.4b46-6) ... 2023-06-23T18:02:27.840 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-06-23T18:02:27.903 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:28.035 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-06-23T18:02:28.054 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-06-23T18:02:28.180 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up indent (2.2.12-1) ... 2023-06-23T18:02:28.331 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-06-23T18:02:28.507 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-06-23T18:02:28.520 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:28.663 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-06-23T18:02:28.666 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-12 (12.15-0ubuntu0.20.04.1) ... 2023-06-23T18:02:28.806 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-06-23T18:02:28.932 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-06-23T18:02:29.056 INFO:teuthology.orchestra.run.smithi169.stdout:Creating new PostgreSQL cluster 12/main ... 2023-06-23T18:02:29.058 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:29.071 INFO:teuthology.orchestra.run.smithi169.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-06-23T18:02:29.117 INFO:teuthology.orchestra.run.smithi169.stdout:The files belonging to this database system will be owned by user "postgres". 2023-06-23T18:02:29.117 INFO:teuthology.orchestra.run.smithi169.stdout:This user must also own the server process. 2023-06-23T18:02:29.118 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:29.118 INFO:teuthology.orchestra.run.smithi169.stdout:The database cluster will be initialized with locale "en_US". 2023-06-23T18:02:29.118 INFO:teuthology.orchestra.run.smithi169.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-06-23T18:02:29.118 INFO:teuthology.orchestra.run.smithi169.stdout:The default text search configuration will be set to "english". 2023-06-23T18:02:29.119 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:29.119 INFO:teuthology.orchestra.run.smithi169.stdout:Data page checksums are disabled. 2023-06-23T18:02:29.119 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:29.119 INFO:teuthology.orchestra.run.smithi169.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-06-23T18:02:29.119 INFO:teuthology.orchestra.run.smithi169.stdout:creating subdirectories ... ok 2023-06-23T18:02:29.120 INFO:teuthology.orchestra.run.smithi169.stdout:selecting dynamic shared memory implementation ... posix 2023-06-23T18:02:29.137 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default max_connections ... 100 2023-06-23T18:02:29.155 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default shared_buffers ... 128MB 2023-06-23T18:02:29.163 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default time zone ... Etc/UTC 2023-06-23T18:02:29.163 INFO:teuthology.orchestra.run.smithi169.stdout:creating configuration files ... ok 2023-06-23T18:02:29.200 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:29.265 INFO:teuthology.orchestra.run.smithi169.stdout:running bootstrap script ... ok 2023-06-23T18:02:29.326 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:29.600 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-06-23T18:02:29.600 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-06-23T18:02:29.776 INFO:teuthology.orchestra.run.smithi169.stdout:performing post-bootstrap initialization ... ok 2023-06-23T18:02:30.113 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-06-23T18:02:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-06-23T18:02:30.348 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-06-23T18:02:30.507 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-06-23T18:02:30.667 INFO:teuthology.orchestra.run.smithi169.stdout:syncing data to disk ... ok 2023-06-23T18:02:30.668 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:30.668 INFO:teuthology.orchestra.run.smithi169.stdout:Success. You can now start the database server using: 2023-06-23T18:02:30.668 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:30.668 INFO:teuthology.orchestra.run.smithi169.stdout: pg_ctlcluster 12 main start 2023-06-23T18:02:30.668 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:30.801 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-06-23T18:02:30.994 INFO:teuthology.orchestra.run.smithi169.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-06-23T18:02:30.994 INFO:teuthology.orchestra.run.smithi169.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-06-23T18:02:31.088 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-06-23T18:02:34.190 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:34.316 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-06-23T18:02:34.517 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-06-23T18:02:34.652 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-06-23T18:02:35.395 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout:The following network interfaces were found in /etc/network/interfaces 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout:which means they are currently configured by ifupdown: 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout:- enp3s0f1 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout:If you want to manage those interfaces with NetworkManager instead 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout:remove their configuration from /etc/network/interfaces. 2023-06-23T18:02:35.396 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:35.545 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-06-23T18:02:35.862 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-06-23T18:02:36.390 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-06-23T18:02:36.879 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-06-23T18:02:37.014 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-06-23T18:02:37.174 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-06-23T18:02:37.275 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-06-23T18:02:37.418 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:37.544 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-06-23T18:02:37.703 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-06-23T18:02:37.964 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-06-23T18:02:37.964 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-06-23T18:02:38.279 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-06-23T18:02:38.405 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-06-23T18:02:38.464 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-06-23T18:02:38.615 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-06-23T18:02:38.741 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-06-23T18:02:38.918 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-06-23T18:02:39.219 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-06-23T18:02:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-06-23T18:02:39.233 DEBUG:teuthology.parallel:result is None 2023-06-23T18:02:46.465 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-06-23T18:02:46.598 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-06-23T18:02:47.559 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-06-23T18:02:47.562 DEBUG:teuthology.parallel:result is None 2023-06-23T18:02:47.562 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:02:47.784 DEBUG:teuthology.orchestra.run.smithi022:> dpkg-query -W -f '${Version}' ceph 2023-06-23T18:02:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:18.0.0-4479-g8627af3e-1focal 2023-06-23T18:02:47.810 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479-g8627af3e-1focal 2023-06-23T18:02:47.811 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479-g8627af3e-1focal is installed. 2023-06-23T18:02:47.812 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:02:48.028 DEBUG:teuthology.orchestra.run.smithi169:> dpkg-query -W -f '${Version}' ceph 2023-06-23T18:02:48.052 INFO:teuthology.orchestra.run.smithi169.stdout:18.0.0-4479-g8627af3e-1focal 2023-06-23T18:02:48.053 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479-g8627af3e-1focal 2023-06-23T18:02:48.053 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479-g8627af3e-1focal is installed. 2023-06-23T18:02:48.054 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T18:02:48.054 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:02:48.071 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:02:48.109 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T18:02:48.110 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:02:48.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:02:48.136 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:02:48.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:02:48.164 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T18:02:48.164 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:02:48.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:02:48.208 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:02:48.224 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:02:48.280 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T18:02:48.281 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:02:48.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:02:48.355 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:02:48.372 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:02:48.388 INFO:teuthology.run_tasks:Running task ceph... 2023-06-23T18:02:48.476 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-23T18:02:48.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2023-06-23T18:02:48.479 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2023-06-23T18:02:48.495 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-23T18:02:48.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T18:02:48.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T18:02:48.551 INFO:tasks.ceph:Creating extra log directories... 2023-06-23T18:02:48.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T18:02:48.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T18:02:48.574 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-23T18:02:48.574 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-23T18:02:48.575 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-23T18:02:48.575 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.575 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T18:02:48.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T18:02:48.619 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.620 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T18:02:48.627 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T18:02:48.716 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-23T18:02:48.717 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-23T18:02:48.717 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313866', 'branch': 'wip-rishabh-improvements-authmon', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7313866', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-improvements-authmon'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-improvements-authmon', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-improvements-authmon', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi022.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '076bbebc42a14f7d568aaa78eabb0038327bcb23', 'timestamp': '2023-06-23_17:37:30', 'tube': 'smithi', 'user': 'rishabh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-23T18:02:48.719 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T18:02:48.722 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T18:02:48.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T18:02:48.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T18:02:48.785 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:48.786 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:02:48.790 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:02:48.790 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2023-06-23T18:02:48.796 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:02:48.796 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:48.796 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-06-23T18:02:48.796 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:48.797 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-06-23 17:58:00.174956652 +0000 2023-06-23T18:02:48.797 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-06-23 17:57:59.866963738 +0000 2023-06-23T18:02:48.797 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-06-23 17:57:59.866963738 +0000 2023-06-23T18:02:48.797 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-06-23T18:02:48.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:02:48.812 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-06-23T18:02:48.812 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-06-23T18:02:48.813 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000267759 s, 1.9 MB/s 2023-06-23T18:02:48.813 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:02:48.821 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 364 Links: 1 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-06-23 17:58:00.666945332 +0000 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-06-23 17:58:00.386951775 +0000 2023-06-23T18:02:48.829 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-06-23 17:58:00.386951775 +0000 2023-06-23T18:02:48.830 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-06-23T18:02:48.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:02:48.842 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-06-23T18:02:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-06-23T18:02:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.00025839 s, 2.0 MB/s 2023-06-23T18:02:48.844 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:02:48.851 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 371 Links: 1 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-06-23 17:58:01.198933093 +0000 2023-06-23T18:02:48.857 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-06-23 17:58:00.898939995 +0000 2023-06-23T18:02:48.858 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-06-23 17:58:00.898939995 +0000 2023-06-23T18:02:48.858 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-06-23T18:02:48.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:02:48.913 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-06-23T18:02:48.914 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-06-23T18:02:48.914 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000293687 s, 1.7 MB/s 2023-06-23T18:02:48.915 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:02:48.922 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2023-06-23T18:02:48.928 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:02:48.928 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:48.928 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-06-23T18:02:48.928 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:48.929 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-06-23 17:58:01.702921498 +0000 2023-06-23T18:02:48.929 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-06-23 17:58:01.398928491 +0000 2023-06-23T18:02:48.929 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-06-23 17:58:01.398928491 +0000 2023-06-23T18:02:48.929 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-06-23T18:02:48.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:02:48.943 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-06-23T18:02:48.943 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-06-23T18:02:48.944 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.00025189 s, 2.0 MB/s 2023-06-23T18:02:48.945 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:02:48.953 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:48.954 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:48.954 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:02:48.961 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:02:48.961 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2023-06-23T18:02:48.968 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 351 Links: 1 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-06-23 17:58:01.930275217 +0000 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-06-23 17:58:01.610282595 +0000 2023-06-23T18:02:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-06-23 17:58:01.610282595 +0000 2023-06-23T18:02:48.970 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-06-23T18:02:48.970 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:02:48.981 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-06-23T18:02:48.982 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-06-23T18:02:48.982 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000417277 s, 1.2 MB/s 2023-06-23T18:02:48.983 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:02:49.034 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2023-06-23T18:02:49.042 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:02:49.042 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:49.042 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-06-23T18:02:49.043 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:49.043 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-06-23 17:58:02.474262666 +0000 2023-06-23T18:02:49.043 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-06-23 17:58:02.170269680 +0000 2023-06-23T18:02:49.043 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-06-23 17:58:02.170269680 +0000 2023-06-23T18:02:49.044 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-06-23T18:02:49.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:02:49.055 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-06-23T18:02:49.055 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-06-23T18:02:49.055 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000406144 s, 1.3 MB/s 2023-06-23T18:02:49.056 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:02:49.106 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2023-06-23T18:02:49.113 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:02:49.113 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:49.113 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-06-23T18:02:49.114 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:49.114 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-06-23 17:58:03.018250126 +0000 2023-06-23T18:02:49.114 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-06-23 17:58:02.722256951 +0000 2023-06-23T18:02:49.115 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-06-23 17:58:02.722256951 +0000 2023-06-23T18:02:49.115 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-06-23T18:02:49.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:02:49.129 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-06-23T18:02:49.129 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-06-23T18:02:49.129 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000330741 s, 1.5 MB/s 2023-06-23T18:02:49.130 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:02:49.182 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 369 Links: 1 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-06-23 17:58:03.518238593 +0000 2023-06-23T18:02:49.189 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-06-23 17:58:03.250244775 +0000 2023-06-23T18:02:49.190 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-06-23 17:58:03.250244775 +0000 2023-06-23T18:02:49.190 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-06-23T18:02:49.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:02:49.202 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-06-23T18:02:49.202 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-06-23T18:02:49.202 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000431751 s, 1.2 MB/s 2023-06-23T18:02:49.203 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:02:49.254 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:49.254 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T18:02:49.254 INFO:tasks.ceph:Generating config... 2023-06-23T18:02:49.256 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-23T18:02:49.256 INFO:tasks.ceph:[client] debug client = 20 2023-06-23T18:02:49.256 INFO:tasks.ceph:[client] debug ms = 1 2023-06-23T18:02:49.257 INFO:tasks.ceph:[client] pid file = 2023-06-23T18:02:49.257 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-23T18:02:49.257 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-23T18:02:49.257 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-23T18:02:49.257 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-23T18:02:49.258 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-23T18:02:49.258 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-23T18:02:49.258 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-23T18:02:49.258 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-23T18:02:49.259 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-23T18:02:49.259 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-23T18:02:49.259 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-23T18:02:49.259 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-23T18:02:49.259 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-23T18:02:49.260 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-23T18:02:49.260 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-23T18:02:49.260 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-23T18:02:49.260 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-23T18:02:49.260 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-23T18:02:49.261 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-23T18:02:49.261 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-23T18:02:49.261 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-06-23T18:02:49.261 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-06-23T18:02:49.261 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-23T18:02:49.262 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-23T18:02:49.262 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-06-23T18:02:49.262 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-23T18:02:49.262 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-23T18:02:49.262 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-23T18:02:49.263 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-23T18:02:49.263 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-23T18:02:49.263 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-23T18:02:49.263 INFO:tasks.ceph:Setting up mon.a... 2023-06-23T18:02:49.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-23T18:02:49.307 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2023-06-23T18:02:49.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-23T18:02:49.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:02:49.379 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2023-06-23T18:02:49.380 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-23T18:02:49.381 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:49.381 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T18:02:49.386 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:49.476 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid e4fadccd-0593-4464-bce3-4c0f7c738354 2023-06-23T18:02:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = pacific 2023-06-23T18:02:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2023-06-23T18:02:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:fsid e4fadccd-0593-4464-bce3-4c0f7c738354 2023-06-23T18:02:49.477 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2023-06-23T18:02:49.475158+0000 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:created 2023-06-23T18:02:49.475158+0000 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 16 (pacific) 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2023-06-23T18:02:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-23T18:02:49.480 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T18:02:49.484 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e4fadccd-0593-4464-bce3-4c0f7c738354... 2023-06-23T18:02:49.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T18:02:49.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T18:02:49.520 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2023-06-23T18:02:49.520 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2023-06-23T18:02:49.520 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2023-06-23T18:02:49.521 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2023-06-23T18:02:49.522 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2023-06-23T18:02:49.523 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2023-06-23T18:02:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = e4fadccd-0593-4464-bce3-4c0f7c738354 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2023-06-23T18:02:49.526 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2023-06-23T18:02:49.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.528 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2023-06-23T18:02:49.529 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 20 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 20 2023-06-23T18:02:49.530 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 10 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2023-06-23T18:02:49.531 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:02:49.532 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-06-23T18:02:49.533 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T18:02:49.534 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.535 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: mon op complaint time = 120 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2023-06-23T18:02:49.536 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: client mount timeout = 600 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-06-23T18:02:49.537 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2023-06-23T18:02:49.538 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2023-06-23T18:02:49.539 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds = 20 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds balancer = 20 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug frag = True 2023-06-23T18:02:49.540 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2023-06-23T18:02:49.541 INFO:teuthology.orchestra.run.smithi022.stdout: mds op complaint time = 180 2023-06-23T18:02:49.541 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2023-06-23T18:02:49.541 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-06-23T18:02:49.541 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-06-23T18:02:49.541 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-06-23T18:02:49.542 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.543 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2023-06-23T18:02:49.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.545 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2023-06-23T18:02:49.546 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2023-06-23T18:02:49.547 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.22,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = e4fadccd-0593-4464-bce3-4c0f7c738354 2023-06-23T18:02:49.548 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2023-06-23T18:02:49.549 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2023-06-23T18:02:49.551 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2023-06-23T18:02:49.552 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 20 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 20 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 10 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2023-06-23T18:02:49.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2023-06-23T18:02:49.554 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2023-06-23T18:02:49.555 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2023-06-23T18:02:49.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.557 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2023-06-23T18:02:49.558 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T18:02:49.559 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2023-06-23T18:02:49.560 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2023-06-23T18:02:49.561 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-06-23T18:02:49.562 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-06-23T18:02:49.563 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-23T18:02:49.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-23T18:02:49.618 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-23T18:02:49.619 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:49.619 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-23T18:02:49.626 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:49.627 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-23T18:02:49.633 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:02:49.633 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:49.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T18:02:49.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:02:49.657 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:02:49.657 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:49.664 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:02:49.665 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:49.665 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T18:02:49.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T18:02:49.694 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:02:49.694 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:49.705 INFO:tasks.ceph:Setting up mon nodes... 2023-06-23T18:02:49.705 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-23T18:02:49.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T18:02:49.758 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T18:02:49.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T18:02:49.813 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T18:02:49.829 INFO:tasks.ceph:Setting up mds nodes... 2023-06-23T18:02:49.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T18:02:49.881 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T18:02:49.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-23T18:02:49.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T18:02:49.942 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T18:02:49.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-23T18:02:49.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T18:02:50.007 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T18:02:50.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-23T18:02:50.023 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T18:02:50.069 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T18:02:50.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-23T18:02:50.083 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-23T18:02:50.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-23T18:02:50.125 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-23T18:02:50.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-23T18:02:50.174 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-23T18:02:50.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-23T18:02:50.230 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-23T18:02:50.243 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-23T18:02:50.281 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-23T18:02:50.295 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-23T18:02:50.295 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T18:02:50.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-23T18:02:50.312 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:50.312 INFO:tasks.ceph:role: osd.0 2023-06-23T18:02:50.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:02:50.312 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T18:02:50.355 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:02:50.355 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:02:50.355 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:02:50.355 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 2023-06-23T18:02:50.355 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:02:50.356 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:02:50.356 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:02:50.356 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:02:50.356 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:02:50.356 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:02:50.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-06-23T18:02:50.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-23T18:02:50.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-23T18:02:50.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:02:50.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:02:50.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-23T18:02:50.448 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:50.448 INFO:tasks.ceph:role: osd.1 2023-06-23T18:02:50.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:02:50.448 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T18:02:50.490 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:02:50.490 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:02:50.491 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:02:50.492 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:02:50.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-06-23T18:02:50.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-23T18:02:50.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-23T18:02:50.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:02:50.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:02:50.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-23T18:02:50.583 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:50.584 INFO:tasks.ceph:role: osd.2 2023-06-23T18:02:50.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:02:50.584 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T18:02:50.624 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:02:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:02:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:02:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 2023-06-23T18:02:50.625 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:02:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:02:50.626 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:02:50.626 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:02:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:02:50.626 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:02:50.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-06-23T18:02:50.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-23T18:02:50.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-23T18:02:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:02:50.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:02:50.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-23T18:02:50.707 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T18:02:50.707 INFO:tasks.ceph:role: osd.3 2023-06-23T18:02:50.707 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:02:50.708 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T18:02:50.750 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:02:50.751 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:02:50.752 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:02:50.752 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:02:50.753 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-06-23T18:02:50.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-23T18:02:50.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-23T18:02:50.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:02:50.831 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:02:50.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-23T18:02:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-23T18:02:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:02:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:02:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T18:02:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:50.913+0000 7f8b5baf7700 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-23T18:02:53.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-23T18:02:53.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-23T18:02:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-23T18:02:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:02:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:02:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T18:02:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:53.977+0000 7f7fc6701700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-23T18:02:56.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-23T18:02:57.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:02:57.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-23T18:02:57.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-23T18:02:57.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:02:57.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:02:57.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T18:02:57.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:02:57.085+0000 7f77d1e04700 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-23T18:03:00.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-23T18:03:00.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-23T18:03:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-23T18:03:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:03:00.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:03:00.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T18:03:00.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:00.133+0000 7f217e028700 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-23T18:03:03.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-23T18:03:03.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-23T18:03:03.206 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:03:03.206 INFO:tasks.ceph:role: osd.4 2023-06-23T18:03:03.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:03:03.207 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T18:03:03.729 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:03:03.729 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:03:03.730 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:03:03.731 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:03:03.732 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-06-23T18:03:03.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-23T18:03:03.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-23T18:03:03.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:03:03.822 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:03:03.823 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-23T18:03:03.833 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:03:03.834 INFO:tasks.ceph:role: osd.5 2023-06-23T18:03:03.834 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:03:03.835 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T18:03:04.417 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:03:04.418 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:03:04.418 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:03:04.418 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-06-23T18:03:04.418 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:03:04.419 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:03:04.419 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:03:04.420 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:03:04.420 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:03:04.420 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:03:04.422 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-06-23T18:03:04.422 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-23T18:03:04.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-23T18:03:04.463 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:03:04.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:03:04.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-23T18:03:04.475 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:03:04.475 INFO:tasks.ceph:role: osd.6 2023-06-23T18:03:04.475 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:03:04.476 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T18:03:04.989 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:03:04.990 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:03:04.991 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:03:04.991 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:03:04.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-06-23T18:03:04.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-23T18:03:05.056 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-23T18:03:05.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:03:05.068 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:03:05.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-23T18:03:05.077 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T18:03:05.077 INFO:tasks.ceph:role: osd.7 2023-06-23T18:03:05.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:03:05.078 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T18:03:05.633 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T18:03:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T18:03:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T18:03:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-06-23T18:03:05.634 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T18:03:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-06-23T18:03:05.635 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T18:03:05.635 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T18:03:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T18:03:05.635 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T18:03:05.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-06-23T18:03:05.636 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-23T18:03:05.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-23T18:03:05.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:03:05.708 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-06-23T18:03:05.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:05.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.832+0000 7f7e003b5700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-23T18:03:05.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.832+0000 7f7e003b5700 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-23T18:03:05.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.832+0000 7f7e003b5700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:03:05.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.832+0000 7f7e003b5700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:03:05.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.832+0000 7f7e003b5700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T18:03:05.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:05.840+0000 7f7e003b5700 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-23T18:03:09.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-23T18:03:09.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:09.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-23T18:03:09.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-23T18:03:09.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:03:09.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:03:09.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T18:03:09.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:09.141+0000 7f049d622700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-23T18:03:12.184 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-23T18:03:12.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:12.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-23T18:03:12.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-23T18:03:12.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:03:12.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:03:12.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T18:03:12.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:12.277+0000 7f667d473700 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-23T18:03:15.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-23T18:03:15.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:15.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-23T18:03:15.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-23T18:03:15.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:03:15.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:03:15.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T18:03:15.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:15.338+0000 7f55bf068700 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-23T18:03:18.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-23T18:03:18.367 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-23T18:03:18.368 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-23T18:03:18.384 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-23T18:03:18.399 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-23T18:03:18.459 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-23T18:03:18.468 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-23T18:03:18.484 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-23T18:03:18.496 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-23T18:03:18.559 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-23T18:03:18.575 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-23T18:03:18.634 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-23T18:03:18.646 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-23T18:03:18.659 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.660 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-23T18:03:18.718 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-23T18:03:18.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:03:18.722 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:03:18.730 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.730 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-23T18:03:18.742 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.742 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-23T18:03:18.749 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:18.750 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-23T18:03:18.829 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.829 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-23T18:03:18.836 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:18.836 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-23T18:03:18.842 INFO:tasks.ceph:Adding keys to all mons... 2023-06-23T18:03:18.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T18:03:18.845 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T18:03:18.889 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVkOlQXLRAAo8mydKBw5tYzhARh+AcIog== 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVk9jlqNBAA0gmLc95g6/e6xHzq2tl3pg== 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVkB2kYOBAAPcbWIvolXCE1AuCCalJxtw== 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2023-06-23T18:03:18.890 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVkQfyfNhAAwByfToNOhTNRZu4O1IBJzg== 2023-06-23T18:03:18.891 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2023-06-23T18:03:18.891 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBN3pVk/T1qOhAAdhEBQ3HwvuCG7H5y6rDR9w== 2023-06-23T18:03:18.891 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2023-06-23T18:03:18.891 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBR3pVk2GU/BRAA7iy4JAvlQYUQnjY8OXfCpg== 2023-06-23T18:03:18.891 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBU3pVkRS4PCBAAoU5OdmhHYLDVe3h3ycNW3g== 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVkzdHILxAAYfEFUbh/pXzqk2Al7jpW4Q== 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVkt5xXOxAAAkvAeEOdEzWHtjoiW/O+kw== 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2023-06-23T18:03:18.892 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVk8K5uAxAAPxXvJ10nVt3jVYdr7KXpVw== 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBZ3pVkk7fOMRAA5uciBvSt10WK5EI4Pe94dA== 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBd3pVkKy2cCBAAWcwFMBYpCQG0uLB+T3fNaQ== 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBg3pVkj0WwEBAAhnZ8ep9E7VgSGSlW/VzHqg== 2023-06-23T18:03:18.893 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBj3pVk69tvFBAAUlxiBT5pCmO4xTdFvS2xWg== 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVksM9cBxAAxr2TtBTZIBuP1bjAu3SXFA== 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout:[client.2] 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVkuTJEChAAAlJ06AT3zJJ5X1pkUPANtw== 2023-06-23T18:03:18.894 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2023-06-23T18:03:18.895 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVk/UgFDRAA9SjI8Pvz0kTFtOxQbylsHg== 2023-06-23T18:03:18.895 INFO:teuthology.orchestra.run.smithi022.stdout:[client.3] 2023-06-23T18:03:18.895 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVkXpoWEBAAxYgwg25L3LtYKfRsc9sI4A== 2023-06-23T18:03:18.900 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBJ3pVkOlQXLRAAo8mydKBw5tYzhARh+AcIog== 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBJ3pVk9jlqNBAA0gmLc95g6/e6xHzq2tl3pg== 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBJ3pVkB2kYOBAAPcbWIvolXCE1AuCCalJxtw== 2023-06-23T18:03:18.901 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVkQfyfNhAAwByfToNOhTNRZu4O1IBJzg== 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBN3pVk/T1qOhAAdhEBQ3HwvuCG7H5y6rDR9w== 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBR3pVk2GU/BRAA7iy4JAvlQYUQnjY8OXfCpg== 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2023-06-23T18:03:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBU3pVkRS4PCBAAoU5OdmhHYLDVe3h3ycNW3g== 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBJ3pVkzdHILxAAYfEFUbh/pXzqk2Al7jpW4Q== 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBJ3pVkt5xXOxAAAkvAeEOdEzWHtjoiW/O+kw== 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVk8K5uAxAAPxXvJ10nVt3jVYdr7KXpVw== 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2023-06-23T18:03:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBZ3pVkk7fOMRAA5uciBvSt10WK5EI4Pe94dA== 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBd3pVkKy2cCBAAWcwFMBYpCQG0uLB+T3fNaQ== 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBg3pVkj0WwEBAAhnZ8ep9E7VgSGSlW/VzHqg== 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBj3pVk69tvFBAAUlxiBT5pCmO4xTdFvS2xWg== 2023-06-23T18:03:18.904 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVksM9cBxAAxr2TtBTZIBuP1bjAu3SXFA== 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVkuTJEChAAAlJ06AT3zJJ5X1pkUPANtw== 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVk/UgFDRAA9SjI8Pvz0kTFtOxQbylsHg== 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2023-06-23T18:03:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBK3pVkXpoWEBAAxYgwg25L3LtYKfRsc9sI4A== 2023-06-23T18:03:18.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:03:18.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:03:18.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:18.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.049 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.301 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:03:19.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T18:03:19.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T18:03:19.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.565 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.617 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T18:03:19.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T18:03:19.931 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-23T18:03:19.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-23T18:03:19.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:03:20.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-23T18:03:20.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-23T18:03:20.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:03:20.114 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-23T18:03:20.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-23T18:03:20.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T18:03:20.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-23T18:03:20.222 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:20.224 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T18:03:20.229 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-23T18:03:20.229 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-23T18:03:20.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-23T18:03:20.232 INFO:tasks.ceph.mon.a:Started 2023-06-23T18:03:20.232 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-23T18:03:20.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-23T18:03:20.234 INFO:tasks.ceph.mon.b:Started 2023-06-23T18:03:20.235 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-23T18:03:20.235 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-23T18:03:20.237 INFO:tasks.ceph.mon.c:Started 2023-06-23T18:03:20.237 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-23T18:03:20.238 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-23T18:03:20.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-23T18:03:20.240 INFO:tasks.ceph.mgr.y:Started 2023-06-23T18:03:20.240 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-23T18:03:20.240 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-23T18:03:20.243 INFO:tasks.ceph.mgr.x:Started 2023-06-23T18:03:20.243 INFO:tasks.ceph:Setting crush tunables to default 2023-06-23T18:03:20.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2023-06-23T18:03:20.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 Processor -- start 2023-06-23T18:03:20.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 -- start start 2023-06-23T18:03:20.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf600b36f0 con 0x7fcf6006c3b0 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf600b0790 con 0x7fcf600b2de0 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.340+0000 7fcf64cd8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6006c820 con 0x7fcf600afdd0 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5dd9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:37038/0 (socket says 172.21.15.22:37038) 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4017936343 learned_addr learned my addr 172.21.15.22:0/4017936343 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5ed9d700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5ed9d700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 msgr2=0x7fcf6015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T18:03:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 msgr2=0x7fcf6015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 msgr2=0x7fcf600b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 msgr2=0x7fcf600b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:03:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.344+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:03:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 msgr2=0x7fcf600b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 msgr2=0x7fcf600b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:03:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 msgr2=0x7fcf6015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 msgr2=0x7fcf6015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5ed9d700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5ed9d700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:03:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.548+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:03:20.789 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:20.780+0000 7f4f574e12c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:03:20.819 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:20.802+0000 7f0eb0afc2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:03:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.22:3300/0 conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5ed9d700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:20.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 msgr2=0x7fcf6015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 -- 172.21.15.22:0/4017936343 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf600b1140 con 0x7fcf6006c3b0 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5dd9b700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf6006c3b0 0x7fcf600b31b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf60071cd0 tx=0x7fcf5400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbe12b7d087f02d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5d59a700 1 -- 172.21.15.22:0/4017936343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54010900 con 0x7fcf6006c3b0 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5d59a700 1 -- 172.21.15.22:0/4017936343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5400c050 con 0x7fcf6006c3b0 2023-06-23T18:03:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf5d59a700 1 -- 172.21.15.22:0/4017936343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54010c60 con 0x7fcf6006c3b0 2023-06-23T18:03:20.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4017936343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf6006c3b0 msgr2=0x7fcf600b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:20.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf6006c3b0 0x7fcf600b31b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf60071cd0 tx=0x7fcf5400ebf0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4017936343 shutdown_connections 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf600b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4017936343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf6006c3b0 0x7fcf600b31b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4017936343 >> v2:172.21.15.169:3300/0 conn(0x7fcf600b2de0 0x7fcf6015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4017936343 >> 172.21.15.22:0/4017936343 conn(0x7fcf60060380 msgr2=0x7fcf60060750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4017936343 shutdown_connections 2023-06-23T18:03:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.952+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4017936343 wait complete. 2023-06-23T18:03:20.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 Processor -- start 2023-06-23T18:03:20.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 -- start start 2023-06-23T18:03:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 0x7fcf601452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf6015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 0x7fcf6015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6006fac0 con 0x7fcf600b2de0 2023-06-23T18:03:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6015c750 con 0x7fcf6006c3b0 2023-06-23T18:03:20.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.956+0000 7fcf64cd8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6015ca30 con 0x7fcf600afdd0 2023-06-23T18:03:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 0x7fcf6015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 0x7fcf601452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5dd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf6015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 0x7fcf6015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53796/0 (socket says 172.21.15.22:53796) 2023-06-23T18:03:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5ed9d700 1 -- 172.21.15.22:0/4127384945 learned_addr learned my addr 172.21.15.22:0/4127384945 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5dd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf6015bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:37996/0 (socket says 172.21.15.22:37996) 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 msgr2=0x7fcf6015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf6015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4127384945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 msgr2=0x7fcf6015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 0x7fcf6015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 -- 172.21.15.22:0/4127384945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5400e570 con 0x7fcf6006c3b0 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf5e59c700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 0x7fcf601452d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf500128e0 tx=0x7fcf50012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004220 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000c050 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6015cd10 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500036c0 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6015d2d0 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcf50005d40 con 0x7fcf6006c3b0 2023-06-23T18:03:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcf50014890 con 0x7fcf6006c3b0 2023-06-23T18:03:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.960+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6004dc70 con 0x7fcf6006c3b0 2023-06-23T18:03:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:20.968+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcf5003db00 con 0x7fcf6006c3b0 2023-06-23T18:03:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.036+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcf600b1cb0 con 0x7fcf6006c3b0 2023-06-23T18:03:21.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.044+0000 7fcf4effd700 1 -- 172.21.15.22:0/4127384945 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcf5002c020 con 0x7fcf6006c3b0 2023-06-23T18:03:21.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 msgr2=0x7fcf601452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 0x7fcf601452d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf500128e0 tx=0x7fcf50012bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 shutdown_connections 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf600b2de0 0x7fcf6015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf600afdd0 0x7fcf6015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 --2- 172.21.15.22:0/4127384945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf6006c3b0 0x7fcf601452d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 >> 172.21.15.22:0/4127384945 conn(0x7fcf60060380 msgr2=0x7fcf600cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 shutdown_connections 2023-06-23T18:03:21.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:21.048+0000 7fcf64cd8700 1 -- 172.21.15.22:0/4127384945 wait complete. 2023-06-23T18:03:21.059 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2023-06-23T18:03:21.071 INFO:tasks.ceph:check_enable_crimson: False 2023-06-23T18:03:21.071 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-23T18:03:21.072 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:21.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-23T18:03:21.085 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:21.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-23T18:03:21.099 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:21.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-23T18:03:21.112 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:21.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-23T18:03:21.123 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:21.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-23T18:03:21.141 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:21.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-23T18:03:21.197 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:21.198 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-23T18:03:21.207 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:03:21.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-23T18:03:21.220 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 91bf5ca8-1094-4e50-969f-08f055225329 0 2023-06-23T18:03:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 Processor -- start 2023-06-23T18:03:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 -- start start 2023-06-23T18:03:21.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4ea00713b0 0x7f4ea0070a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea0070fa0 con 0x7f4ea006a2b0 2023-06-23T18:03:21.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00711d0 con 0x7f4ea006e750 2023-06-23T18:03:21.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.315+0000 7f4ea58a0700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea015df30 con 0x7f4ea00713b0 2023-06-23T18:03:21.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea509f700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4ea006e750 0x7f4ea00caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:43746/0 (socket says 172.21.15.169:43746) 2023-06-23T18:03:21.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea509f700 1 -- 172.21.15.169:0/4091326392 learned_addr learned my addr 172.21.15.169:0/4091326392 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-06-23T18:03:21.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9ffff700 1 --2- 172.21.15.169:0/4091326392 >> v2:172.21.15.22:3300/0 conn(0x7f4ea00713b0 0x7f4ea0070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372905199 0 0) 0x7f4ea00711d0 con 0x7f4ea006e750 2023-06-23T18:03:21.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e900035d0 con 0x7f4ea006e750 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122412306 0 0) 0x7f4e900035d0 con 0x7f4ea006e750 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9ffff700 1 -- 172.21.15.169:0/4091326392 >> v1:172.21.15.169:6789/0 conn(0x7f4ea006e750 legacy=0x7f4ea00caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9ffff700 1 -- 172.21.15.169:0/4091326392 >> v1:172.21.15.22:6789/0 conn(0x7f4ea006a2b0 legacy=0x7f4ea015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9ffff700 1 -- 172.21.15.169:0/4091326392 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea015a130 con 0x7f4ea00713b0 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9ffff700 1 --2- 172.21.15.169:0/4091326392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea00713b0 0x7f4ea0070a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097d0 tx=0x7f4e8c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba5f43de9f5b887 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980019f0 con 0x7f4ea00713b0 2023-06-23T18:03:21.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c004030 con 0x7f4ea00713b0 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4e9f7fe700 1 -- 172.21.15.169:0/4091326392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0043c0 con 0x7f4ea00713b0 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 -- 172.21.15.169:0/4091326392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea00713b0 msgr2=0x7f4ea0070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/4091326392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea00713b0 0x7f4ea0070a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097d0 tx=0x7f4e8c009ae0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 -- 172.21.15.169:0/4091326392 shutdown_connections 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/4091326392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea00713b0 0x7f4ea0070a60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 -- 172.21.15.169:0/4091326392 >> 172.21.15.169:0/4091326392 conn(0x7f4ea00cc820 msgr2=0x7f4ea00cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 -- 172.21.15.169:0/4091326392 shutdown_connections 2023-06-23T18:03:21.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.319+0000 7f4ea58a0700 1 -- 172.21.15.169:0/4091326392 wait complete. 2023-06-23T18:03:21.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 Processor -- start 2023-06-23T18:03:21.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- start start 2023-06-23T18:03:21.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea006e750 0x7f4ea00c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 0x7f4ea00aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00c4b70 con 0x7f4ea006e750 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00c3590 con 0x7f4ea00713b0 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00c3870 con 0x7f4ea006a2b0 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea489e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea489e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33492/0 (socket says 172.21.15.169:33492) 2023-06-23T18:03:21.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea489e700 1 -- 172.21.15.169:0/688363587 learned_addr learned my addr 172.21.15.169:0/688363587 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 0x7f4ea00aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4e9ffff700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea006e750 0x7f4ea00c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 -- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 msgr2=0x7f4ea00c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 -- 172.21.15.169:0/688363587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea006e750 msgr2=0x7f4ea00c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea006e750 0x7f4ea00c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 -- 172.21.15.169:0/688363587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0094b0 con 0x7f4ea00713b0 2023-06-23T18:03:21.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea489e700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:21.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea509f700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 0x7f4ea00aecb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800cb80 tx=0x7f4e9800ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800aaa0 con 0x7f4ea00713b0 2023-06-23T18:03:21.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800ac00 con 0x7f4ea00713b0 2023-06-23T18:03:21.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea00c3b50 con 0x7f4ea00713b0 2023-06-23T18:03:21.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980070d0 con 0x7f4ea00713b0 2023-06-23T18:03:21.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.323+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea00c4110 con 0x7f4ea00713b0 2023-06-23T18:03:21.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.327+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e98007530 con 0x7f4ea00713b0 2023-06-23T18:03:21.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.327+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4e9801dcc0 con 0x7f4ea00713b0 2023-06-23T18:03:21.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.327+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea00c39b0 con 0x7f4ea00713b0 2023-06-23T18:03:21.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.327+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4ea00c39b0 con 0x7f4ea00713b0 2023-06-23T18:03:21.389 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:21.380+0000 7f4f574e12c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:03:21.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.407+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "91bf5ca8-1094-4e50-969f-08f055225329", "id": 0} v 0) v1 -- 0x7f4ea004e4c0 con 0x7f4ea00713b0 2023-06-23T18:03:21.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4e9d7fa700 1 -- 172.21.15.169:0/688363587 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91bf5ca8-1094-4e50-969f-08f055225329", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e98018070 con 0x7f4ea00713b0 2023-06-23T18:03:21.432 INFO:teuthology.orchestra.run.smithi169.stdout:0 2023-06-23T18:03:21.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 msgr2=0x7f4ea00aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 0x7f4ea00aecb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800cb80 tx=0x7f4e9800ce90 comp rx=0 tx=0).stop 2023-06-23T18:03:21.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 shutdown_connections 2023-06-23T18:03:21.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea00713b0 0x7f4ea00aecb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ea006e750 0x7f4ea00c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 --2- 172.21.15.169:0/688363587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea006a2b0 0x7f4ea00c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 >> 172.21.15.169:0/688363587 conn(0x7f4ea00cc820 msgr2=0x7f4ea00b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 shutdown_connections 2023-06-23T18:03:21.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.419+0000 7f4ea58a0700 1 -- 172.21.15.169:0/688363587 wait complete. 2023-06-23T18:03:21.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new bb186bf3-9d2b-4443-bf14-011526a0d3ed 1 2023-06-23T18:03:21.446 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:21.436+0000 7f4f574e12c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:03:21.475 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:21.463+0000 7f0eb0afc2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:03:21.535 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:21.523+0000 7f0eb0afc2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:03:21.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.543+0000 7f9fd8967700 1 Processor -- start 2023-06-23T18:03:21.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- start start 2023-06-23T18:03:21.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 0x7f9fd406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 0x7f9fd40b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd406eab0 con 0x7f9fd4158b20 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd40cdf20 con 0x7f9fd4070e50 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40cd680 con 0x7f9fd4158ef0 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 0x7f9fd406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd2d9d700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9fd4158b20 0x7f9fd4070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:43748/0 (socket says 172.21.15.169:43748) 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd2d9d700 1 -- 172.21.15.169:0/2247559994 learned_addr learned my addr 172.21.15.169:0/2247559994 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-06-23T18:03:21.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd359e700 1 --2- 172.21.15.169:0/2247559994 >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 0x7f9fd40b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 -- 172.21.15.169:0/2247559994 >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 msgr2=0x7f9fd40b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:21.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 --2- 172.21.15.169:0/2247559994 >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 0x7f9fd40b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 -- 172.21.15.169:0/2247559994 >> v1:172.21.15.169:6789/0 conn(0x7f9fd4158b20 legacy=0x7f9fd4070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 -- 172.21.15.169:0/2247559994 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd40cd960 con 0x7f9fd4158ef0 2023-06-23T18:03:21.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd1d9b700 1 -- 172.21.15.169:0/2247559994 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222566187 0 0) 0x7f9fd406eab0 con 0x7f9fd4158b20 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd359e700 1 --2- 172.21.15.169:0/2247559994 >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 0x7f9fd40b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd259c700 1 --2- 172.21.15.169:0/2247559994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 0x7f9fd406e440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009850 tx=0x7f9fc8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fea7af796b69135 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd1d9b700 1 -- 172.21.15.169:0/2247559994 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8011830 con 0x7f9fd4158ef0 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd1d9b700 1 -- 172.21.15.169:0/2247559994 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc8011990 con 0x7f9fd4158ef0 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd1d9b700 1 -- 172.21.15.169:0/2247559994 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8011d20 con 0x7f9fd4158ef0 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- 172.21.15.169:0/2247559994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 msgr2=0x7f9fd406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 --2- 172.21.15.169:0/2247559994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 0x7f9fd406e440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009850 tx=0x7f9fc8009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- 172.21.15.169:0/2247559994 shutdown_connections 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 --2- 172.21.15.169:0/2247559994 >> v2:172.21.15.22:3300/0 conn(0x7f9fd4070e50 0x7f9fd40b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 --2- 172.21.15.169:0/2247559994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4158ef0 0x7f9fd406e440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- 172.21.15.169:0/2247559994 >> 172.21.15.169:0/2247559994 conn(0x7f9fd406b700 msgr2=0x7f9fd40b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- 172.21.15.169:0/2247559994 shutdown_connections 2023-06-23T18:03:21.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.547+0000 7f9fd8967700 1 -- 172.21.15.169:0/2247559994 wait complete. 2023-06-23T18:03:21.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 Processor -- start 2023-06-23T18:03:21.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 -- start start 2023-06-23T18:03:21.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 0x7f9fd41ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 0x7f9fd40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40cbb50 con 0x7f9fd40cb170 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41ee550 con 0x7f9fd4158b20 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd8967700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd415b4f0 con 0x7f9fd4070e50 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd2d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 0x7f9fd41ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45406/0 (socket says 172.21.15.169:45406) 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd2d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 0x7f9fd41ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33500/0 (socket says 172.21.15.169:33500) 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 -- 172.21.15.169:0/131108214 learned_addr learned my addr 172.21.15.169:0/131108214 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd359e700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 0x7f9fd40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 -- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 msgr2=0x7f9fd41ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 0x7f9fd41ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 -- 172.21.15.169:0/131108214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 msgr2=0x7f9fd40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 0x7f9fd40cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd259c700 1 -- 172.21.15.169:0/131108214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80094b0 con 0x7f9fd4158b20 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.551+0000 7f9fd359e700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 0x7f9fd40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fd259c700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000c00 tx=0x7f9fc8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80045d0 con 0x7f9fd4158b20 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc801f070 con 0x7f9fd4158b20 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd415b7d0 con 0x7f9fd4158b20 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8003900 con 0x7f9fd4158b20 2023-06-23T18:03:21.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd415bd90 con 0x7f9fd4158b20 2023-06-23T18:03:21.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9fc801a430 con 0x7f9fd4158b20 2023-06-23T18:03:21.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.555+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f9fc8021070 con 0x7f9fd4158b20 2023-06-23T18:03:21.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.559+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4071250 con 0x7f9fd4158b20 2023-06-23T18:03:21.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.559+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9fc802a020 con 0x7f9fd4158b20 2023-06-23T18:03:21.589 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:21.580+0000 7f4f574e12c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:03:21.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.639+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bb186bf3-9d2b-4443-bf14-011526a0d3ed", "id": 1} v 0) v1 -- 0x7f9fd404e4c0 con 0x7f9fd4158b20 2023-06-23T18:03:21.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.647+0000 7f9fbb7fe700 1 -- 172.21.15.169:0/131108214 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb186bf3-9d2b-4443-bf14-011526a0d3ed", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9fc8015020 con 0x7f9fd4158b20 2023-06-23T18:03:21.662 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-06-23T18:03:21.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 msgr2=0x7f9fd414c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000c00 tx=0x7f9fc8004770 comp rx=0 tx=0).stop 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 shutdown_connections 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fd40cb170 0x7f9fd40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4158b20 0x7f9fd414c870 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 --2- 172.21.15.169:0/131108214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9fd4070e50 0x7f9fd41ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 >> 172.21.15.169:0/131108214 conn(0x7f9fd406b700 msgr2=0x7f9fd4154870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 shutdown_connections 2023-06-23T18:03:21.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.651+0000 7f9fd8967700 1 -- 172.21.15.169:0/131108214 wait complete. 2023-06-23T18:03:21.678 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 230da105-5aaa-46f0-85dd-38990a357b51 2 2023-06-23T18:03:21.695 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:21.683+0000 7f0eb0afc2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:03:21.792 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:21.784+0000 7f4f574e12c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:03:21.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.791+0000 7f66c971d700 1 Processor -- start 2023-06-23T18:03:21.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.791+0000 7f66c971d700 1 -- start start 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66c40b0bf0 0x7f66c406fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40ce6e0 con 0x7f66c40cefa0 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c406af50 con 0x7f66c40b0bf0 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c415f900 con 0x7f66c415be20 2023-06-23T18:03:21.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66c40b0bf0 0x7f66c406fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33506/0 (socket says 172.21.15.169:33506) 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66c40b0bf0 0x7f66c406fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45410/0 (socket says 172.21.15.169:45410) 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c3fff700 1 -- 172.21.15.169:0/545013122 learned_addr learned my addr 172.21.15.169:0/545013122 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 -- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 msgr2=0x7f66c406aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 -- 172.21.15.169:0/545013122 >> v1:172.21.15.22:6789/0 conn(0x7f66c40cefa0 legacy=0x7f66c40b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 -- 172.21.15.169:0/545013122 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c415fbe0 con 0x7f66c40b0bf0 2023-06-23T18:03:21.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c3fff700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:21.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c8f1c700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c40b0bf0 0x7f66c406fe20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009d90 tx=0x7f66bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7e604da7d395ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c2ffd700 1 -- 172.21.15.169:0/545013122 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00e070 con 0x7f66c40b0bf0 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c2ffd700 1 -- 172.21.15.169:0/545013122 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00bb20 con 0x7f66c40b0bf0 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c2ffd700 1 -- 172.21.15.169:0/545013122 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012430 con 0x7f66c40b0bf0 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c40b0bf0 msgr2=0x7f66c406fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c40b0bf0 0x7f66c406fe20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009d90 tx=0x7f66bc00b330 comp rx=0 tx=0).stop 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- 172.21.15.169:0/545013122 shutdown_connections 2023-06-23T18:03:21.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c40b0bf0 0x7f66c406fe20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 --2- 172.21.15.169:0/545013122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c406aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- 172.21.15.169:0/545013122 >> 172.21.15.169:0/545013122 conn(0x7f66c406ca60 msgr2=0x7f66c40ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- 172.21.15.169:0/545013122 shutdown_connections 2023-06-23T18:03:21.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- 172.21.15.169:0/545013122 wait complete. 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 Processor -- start 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.795+0000 7f66c971d700 1 -- start start 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 0x7f66c41f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41529d0 con 0x7f66c40cefa0 2023-06-23T18:03:21.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41f1070 con 0x7f66c4153270 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4159c20 con 0x7f66c415be20 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45416/0 (socket says 172.21.15.169:45416) 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 -- 172.21.15.169:0/676996637 learned_addr learned my addr 172.21.15.169:0/676996637 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c3fff700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 0x7f66c41f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 -- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 msgr2=0x7f66c4152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c4152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 -- 172.21.15.169:0/676996637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 msgr2=0x7f66c41f0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 0x7f66c41f0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 -- 172.21.15.169:0/676996637 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8009570 con 0x7f66c4153270 2023-06-23T18:03:21.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c37fe700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:21.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c3fff700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 0x7f66c41f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:21.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c8f1c700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66bc014040 tx=0x7f66bc004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:21.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012430 con 0x7f66c4153270 2023-06-23T18:03:21.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc020070 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc009a70 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004280 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c415cd80 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f66bc0046e0 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f66bc022070 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.799+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c4159e20 con 0x7f66c4153270 2023-06-23T18:03:21.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.803+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f66bc045b00 con 0x7f66c4153270 2023-06-23T18:03:21.845 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:21.836+0000 7f4f574e12c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:03:21.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.899+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "230da105-5aaa-46f0-85dd-38990a357b51", "id": 2} v 0) v1 -- 0x7f66c404e4c0 con 0x7f66c4153270 2023-06-23T18:03:21.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c0ff9700 1 -- 172.21.15.169:0/676996637 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "230da105-5aaa-46f0-85dd-38990a357b51", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f66bc034030 con 0x7f66c4153270 2023-06-23T18:03:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:2 2023-06-23T18:03:21.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 msgr2=0x7f66c415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66bc014040 tx=0x7f66bc004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 shutdown_connections 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c4153270 0x7f66c415d910 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c415be20 0x7f66c4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 --2- 172.21.15.169:0/676996637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c40cefa0 0x7f66c41f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 >> 172.21.15.169:0/676996637 conn(0x7f66c40b0bf0 msgr2=0x7f66c41556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:21.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 shutdown_connections 2023-06-23T18:03:21.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f66c971d700 1 -- 172.21.15.169:0/676996637 wait complete. 2023-06-23T18:03:21.922 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:21.907+0000 7f0eb0afc2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:03:21.932 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 81d76dd6-d216-41d6-991e-b4c32bea0c3c 3 2023-06-23T18:03:21.972 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:21.959+0000 7f0eb0afc2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:03:22.020 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.008+0000 7f4f574e12c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:03:22.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.027+0000 7f5836346700 1 Processor -- start 2023-06-23T18:03:22.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.027+0000 7f5836346700 1 -- start start 2023-06-23T18:03:22.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f58300cd8e0 0x7f58300cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583015a700 con 0x7f5830159c30 2023-06-23T18:03:22.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58300b2720 con 0x7f58300cd8e0 2023-06-23T18:03:22.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58300ce7b0 con 0x7f58300b1d60 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33526/0 (socket says 172.21.15.169:33526) 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 -- 172.21.15.169:0/2514884914 learned_addr learned my addr 172.21.15.169:0/2514884914 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5834b43700 1 --2- 172.21.15.169:0/2514884914 >> v2:172.21.15.22:3300/0 conn(0x7f58300cd8e0 0x7f58300cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 -- 172.21.15.169:0/2514884914 >> v2:172.21.15.22:3300/0 conn(0x7f58300cd8e0 msgr2=0x7f58300cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 --2- 172.21.15.169:0/2514884914 >> v2:172.21.15.22:3300/0 conn(0x7f58300cd8e0 0x7f58300cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 -- 172.21.15.169:0/2514884914 >> v1:172.21.15.22:6789/0 conn(0x7f5830159c30 legacy=0x7f583015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 -- 172.21.15.169:0/2514884914 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300cea90 con 0x7f58300b1d60 2023-06-23T18:03:22.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5835b45700 1 --2- 172.21.15.169:0/2514884914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5828009e50 tx=0x7f582800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8161ae38e2659616 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5827fff700 1 -- 172.21.15.169:0/2514884914 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582800e070 con 0x7f58300b1d60 2023-06-23T18:03:22.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5827fff700 1 -- 172.21.15.169:0/2514884914 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828003a40 con 0x7f58300b1d60 2023-06-23T18:03:22.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5827fff700 1 -- 172.21.15.169:0/2514884914 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828003df0 con 0x7f58300b1d60 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- 172.21.15.169:0/2514884914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 msgr2=0x7f58300b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 --2- 172.21.15.169:0/2514884914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5828009e50 tx=0x7f582800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- 172.21.15.169:0/2514884914 shutdown_connections 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 --2- 172.21.15.169:0/2514884914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b21e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 --2- 172.21.15.169:0/2514884914 >> v2:172.21.15.22:3300/0 conn(0x7f58300cd8e0 0x7f58300cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- 172.21.15.169:0/2514884914 >> 172.21.15.169:0/2514884914 conn(0x7f58300ccd60 msgr2=0x7f58300cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- 172.21.15.169:0/2514884914 shutdown_connections 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- 172.21.15.169:0/2514884914 wait complete. 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 Processor -- start 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.031+0000 7f5836346700 1 -- start start 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 0x7f58301ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5830159c30 0x7f583013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830157660 con 0x7f5830159c30 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830157940 con 0x7f58300cd8e0 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5830157c20 con 0x7f58300b1d60 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5834b43700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 0x7f58301ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33528/0 (socket says 172.21.15.169:33528) 2023-06-23T18:03:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5834b43700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 0x7f58301ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45418/0 (socket says 172.21.15.169:45418) 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 -- 172.21.15.169:0/1251079847 learned_addr learned my addr 172.21.15.169:0/1251079847 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835b45700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5830159c30 0x7f583013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 -- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 msgr2=0x7f58301ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 0x7f58301ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 -- 172.21.15.169:0/1251079847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5830159c30 msgr2=0x7f583013f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5830159c30 0x7f583013f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 -- 172.21.15.169:0/1251079847 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828009b30 con 0x7f58300b1d60 2023-06-23T18:03:22.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5835344700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f582c00b4a0 tx=0x7f582c00b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c011830 con 0x7f58300b1d60 2023-06-23T18:03:22.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c011990 con 0x7f58300b1d60 2023-06-23T18:03:22.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830157f00 con 0x7f58300b1d60 2023-06-23T18:03:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c011d40 con 0x7f58300b1d60 2023-06-23T18:03:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5830158420 con 0x7f58300b1d60 2023-06-23T18:03:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f582c004320 con 0x7f58300b1d60 2023-06-23T18:03:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f582c01a890 con 0x7f58300b1d60 2023-06-23T18:03:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.035+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58301580f0 con 0x7f58300b1d60 2023-06-23T18:03:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.039+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f582c038b00 con 0x7f58300b1d60 2023-06-23T18:03:22.086 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.076+0000 7f4f574e12c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:03:22.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.123+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81d76dd6-d216-41d6-991e-b4c32bea0c3c", "id": 3} v 0) v1 -- 0x7f583004e4c0 con 0x7f58300b1d60 2023-06-23T18:03:22.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5825ffb700 1 -- 172.21.15.169:0/1251079847 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81d76dd6-d216-41d6-991e-b4c32bea0c3c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f582c027070 con 0x7f58300b1d60 2023-06-23T18:03:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:3 2023-06-23T18:03:22.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 msgr2=0x7f58300b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f582c00b4a0 tx=0x7f582c00b7b0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 shutdown_connections 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5830159c30 0x7f583013f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58300cd8e0 0x7f58301ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 --2- 172.21.15.169:0/1251079847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58300b1d60 0x7f58300b57a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 >> 172.21.15.169:0/1251079847 conn(0x7f58300ccd60 msgr2=0x7f58300caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 shutdown_connections 2023-06-23T18:03:22.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.131+0000 7f5836346700 1 -- 172.21.15.169:0/1251079847 wait complete. 2023-06-23T18:03:22.147 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:22.135+0000 7f0eb0afc2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:03:22.151 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.140+0000 7f4f574e12c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:03:22.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new f344bea8-74a9-462e-b206-9d6a1d53b973 4 2023-06-23T18:03:22.212 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:22.199+0000 7f0eb0afc2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:03:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.247+0000 7f1915afa700 1 Processor -- start 2023-06-23T18:03:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.247+0000 7f1915afa700 1 -- start start 2023-06-23T18:03:22.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 0x7f19100c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19100b0b30 0x7f19100c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100cdce0 con 0x7f19100b1080 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191006b210 con 0x7f1910157ef0 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100cdf80 con 0x7f19100b0b30 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190ffff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f19100b1080 0x7f191015bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:43764/0 (socket says 172.21.15.169:43764) 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190ffff700 1 -- 172.21.15.169:0/2932116274 learned_addr learned my addr 172.21.15.169:0/2932116274 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-06-23T18:03:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 --2- 172.21.15.169:0/2932116274 >> v2:172.21.15.169:3300/0 conn(0x7f19100b0b30 0x7f19100c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1914af8700 1 --2- 172.21.15.169:0/2932116274 >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 0x7f19100c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190f7fe700 1 -- 172.21.15.169:0/2932116274 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559348259 0 0) 0x7f19100cdce0 con 0x7f19100b1080 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190f7fe700 1 -- 172.21.15.169:0/2932116274 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19000046e0 con 0x7f19100b1080 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 -- 172.21.15.169:0/2932116274 >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 msgr2=0x7f19100c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 --2- 172.21.15.169:0/2932116274 >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 0x7f19100c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 -- 172.21.15.169:0/2932116274 >> v1:172.21.15.169:6789/0 conn(0x7f19100b1080 legacy=0x7f191015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 -- 172.21.15.169:0/2932116274 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100ce260 con 0x7f19100b0b30 2023-06-23T18:03:22.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1914af8700 1 --2- 172.21.15.169:0/2932116274 >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 0x7f19100c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:22.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f19152f9700 1 --2- 172.21.15.169:0/2932116274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f19100c9f90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f19100b1530 tx=0x7f190800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc27409e796f69b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190f7fe700 1 -- 172.21.15.169:0/2932116274 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800e070 con 0x7f19100b0b30 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190f7fe700 1 -- 172.21.15.169:0/2932116274 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190800bb20 con 0x7f19100b0b30 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f190f7fe700 1 -- 172.21.15.169:0/2932116274 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908012430 con 0x7f19100b0b30 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- 172.21.15.169:0/2932116274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 msgr2=0x7f19100c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 --2- 172.21.15.169:0/2932116274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f19100c9f90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f19100b1530 tx=0x7f190800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- 172.21.15.169:0/2932116274 shutdown_connections 2023-06-23T18:03:22.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 --2- 172.21.15.169:0/2932116274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f19100c9f90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 --2- 172.21.15.169:0/2932116274 >> v2:172.21.15.22:3300/0 conn(0x7f1910157ef0 0x7f19100c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- 172.21.15.169:0/2932116274 >> 172.21.15.169:0/2932116274 conn(0x7f191006c9d0 msgr2=0x7f19100ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- 172.21.15.169:0/2932116274 shutdown_connections 2023-06-23T18:03:22.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.251+0000 7f1915afa700 1 -- 172.21.15.169:0/2932116274 wait complete. 2023-06-23T18:03:22.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 Processor -- start 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- start start 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19100b1080 0x7f19100c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1910157ef0 0x7f191014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100c7350 con 0x7f1910157ef0 2023-06-23T18:03:22.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101503f0 con 0x7f19100b0b30 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100717a0 con 0x7f19100b1080 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45432/0 (socket says 172.21.15.169:45432) 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 -- 172.21.15.169:0/2900382436 learned_addr learned my addr 172.21.15.169:0/2900382436 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f190ffff700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19100b1080 0x7f19100c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f19152f9700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1910157ef0 0x7f191014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 -- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19100b1080 msgr2=0x7f19100c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19100b1080 0x7f19100c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 -- 172.21.15.169:0/2900382436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1910157ef0 msgr2=0x7f191014feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1910157ef0 0x7f191014feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 -- 172.21.15.169:0/2900382436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040094b0 con 0x7f19100b0b30 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1914af8700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f190400ebe0 tx=0x7f190400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400c850 con 0x7f19100b0b30 2023-06-23T18:03:22.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190400c9b0 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1908009a70 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400cd40 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.255+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1910071d60 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.259+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1904005d40 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.259+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1910071f80 con 0x7f19100b0b30 2023-06-23T18:03:22.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.259+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f1904006910 con 0x7f19100b0b30 2023-06-23T18:03:22.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.263+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1904021060 con 0x7f19100b0b30 2023-06-23T18:03:22.278 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:22.263+0000 7f0eb0afc2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:03:22.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.339+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f344bea8-74a9-462e-b206-9d6a1d53b973", "id": 4} v 0) v1 -- 0x7f191004e4c0 con 0x7f19100b0b30 2023-06-23T18:03:22.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f190d7fa700 1 -- 172.21.15.169:0/2900382436 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f344bea8-74a9-462e-b206-9d6a1d53b973", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f19040290b0 con 0x7f19100b0b30 2023-06-23T18:03:22.360 INFO:teuthology.orchestra.run.smithi169.stdout:4 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 msgr2=0x7f1910150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f190400ebe0 tx=0x7f190400eef0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 shutdown_connections 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1910157ef0 0x7f191014feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19100b1080 0x7f19100c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 --2- 172.21.15.169:0/2900382436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19100b0b30 0x7f1910150990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 >> 172.21.15.169:0/2900382436 conn(0x7f191006c9d0 msgr2=0x7f19100b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 shutdown_connections 2023-06-23T18:03:22.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.347+0000 7f1915afa700 1 -- 172.21.15.169:0/2900382436 wait complete. 2023-06-23T18:03:22.372 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 9ac9c71a-a05a-4486-a6bd-9870e66f848a 5 2023-06-23T18:03:22.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.463+0000 7f4d36875700 1 Processor -- start 2023-06-23T18:03:22.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.463+0000 7f4d36875700 1 -- start start 2023-06-23T18:03:22.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d300b1920 0x7f4d300cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300cca20 0x7f4d300d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300d1d40 con 0x7f4d300d0a20 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d300b1cf0 con 0x7f4d300b1920 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300d31e0 con 0x7f4d300cca20 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d300b1920 0x7f4d300cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d300b1920 0x7f4d300cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45444/0 (socket says 172.21.15.169:45444) 2023-06-23T18:03:22.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 -- 172.21.15.169:0/3639495785 learned_addr learned my addr 172.21.15.169:0/3639495785 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36074700 1 --1- 172.21.15.169:0/3639495785 >> v1:172.21.15.169:6789/0 conn(0x7f4d300d0a20 0x7f4d3015f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:43770/0 (socket says 172.21.15.169:43770) 2023-06-23T18:03:22.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35072700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300cca20 0x7f4d300d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d34871700 1 -- 172.21.15.169:0/3639495785 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303450956 0 0) 0x7f4d300d1d40 con 0x7f4d300d0a20 2023-06-23T18:03:22.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d34871700 1 -- 172.21.15.169:0/3639495785 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d1c0035d0 con 0x7f4d300d0a20 2023-06-23T18:03:22.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 -- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300cca20 msgr2=0x7f4d300d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300cca20 0x7f4d300d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 -- 172.21.15.169:0/3639495785 >> v1:172.21.15.169:6789/0 conn(0x7f4d300d0a20 legacy=0x7f4d3015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 -- 172.21.15.169:0/3639495785 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300cf130 con 0x7f4d300b1920 2023-06-23T18:03:22.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d35873700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300b1920 0x7f4d300cffa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e50 tx=0x7f4d2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a4ec9fb0c40832 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d34871700 1 -- 172.21.15.169:0/3639495785 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800e070 con 0x7f4d300b1920 2023-06-23T18:03:22.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d34871700 1 -- 172.21.15.169:0/3639495785 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2800b910 con 0x7f4d300b1920 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d34871700 1 -- 172.21.15.169:0/3639495785 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800bcc0 con 0x7f4d300b1920 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300b1920 msgr2=0x7f4d300cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300b1920 0x7f4d300cffa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e50 tx=0x7f4d2800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- 172.21.15.169:0/3639495785 shutdown_connections 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300cca20 0x7f4d300d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 --2- 172.21.15.169:0/3639495785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300b1920 0x7f4d300cffa0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- 172.21.15.169:0/3639495785 >> 172.21.15.169:0/3639495785 conn(0x7f4d30069960 msgr2=0x7f4d30069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- 172.21.15.169:0/3639495785 shutdown_connections 2023-06-23T18:03:22.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.467+0000 7f4d36875700 1 -- 172.21.15.169:0/3639495785 wait complete. 2023-06-23T18:03:22.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 Processor -- start 2023-06-23T18:03:22.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 -- start start 2023-06-23T18:03:22.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 0x7f4d301507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300d0a20 0x7f4d300b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30152d70 con 0x7f4d300cca20 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30152440 con 0x7f4d300d0a20 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36875700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30152720 con 0x7f4d300b1920 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33554/0 (socket says 172.21.15.169:33554) 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d36074700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300d0a20 0x7f4d300b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 -- 172.21.15.169:0/2116504942 learned_addr learned my addr 172.21.15.169:0/2116504942 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35072700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 0x7f4d301507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 -- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300d0a20 msgr2=0x7f4d300b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300d0a20 0x7f4d300b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 -- 172.21.15.169:0/2116504942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 msgr2=0x7f4d301507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 0x7f4d301507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 -- 172.21.15.169:0/2116504942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28009b30 con 0x7f4d300b1920 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35072700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 0x7f4d301507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:22.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.471+0000 7f4d35873700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b370 tx=0x7f4d28004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d280048b0 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d280092e0 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d30152a00 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2801c680 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d30150d20 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4d2801cb60 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f4d28003e30 con 0x7f4d300b1920 2023-06-23T18:03:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.475+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3004e4c0 con 0x7f4d300b1920 2023-06-23T18:03:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.479+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4d28044b00 con 0x7f4d300b1920 2023-06-23T18:03:22.564 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.552+0000 7f4f574e12c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:03:22.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.555+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9ac9c71a-a05a-4486-a6bd-9870e66f848a", "id": 5} v 0) v1 -- 0x7f4d300b46d0 con 0x7f4d300b1920 2023-06-23T18:03:22.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d267fc700 1 -- 172.21.15.169:0/2116504942 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ac9c71a-a05a-4486-a6bd-9870e66f848a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4d28033070 con 0x7f4d300b1920 2023-06-23T18:03:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:5 2023-06-23T18:03:22.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 msgr2=0x7f4d300c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b370 tx=0x7f4d28004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 shutdown_connections 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d300d0a20 0x7f4d300b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300cca20 0x7f4d301507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 --2- 172.21.15.169:0/2116504942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d300b1920 0x7f4d300c6ca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 >> 172.21.15.169:0/2116504942 conn(0x7f4d30069960 msgr2=0x7f4d300b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 shutdown_connections 2023-06-23T18:03:22.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.563+0000 7f4d36875700 1 -- 172.21.15.169:0/2116504942 wait complete. 2023-06-23T18:03:22.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05 6 2023-06-23T18:03:22.688 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.676+0000 7f4f574e12c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:03:22.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.679+0000 7f267759e700 1 Processor -- start 2023-06-23T18:03:22.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.679+0000 7f267759e700 1 -- start start 2023-06-23T18:03:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2678159cd0 0x7f26780b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f267806e6e0 0x7f267806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267815a0a0 con 0x7f267806da30 2023-06-23T18:03:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780b1740 con 0x7f2678159cd0 2023-06-23T18:03:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780cae40 con 0x7f267806e6e0 2023-06-23T18:03:22.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f267806e6e0 0x7f267806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f267806e6e0 0x7f267806eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45448/0 (socket says 172.21.15.169:45448) 2023-06-23T18:03:22.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 -- 172.21.15.169:0/181706855 learned_addr learned my addr 172.21.15.169:0/181706855 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267659c700 1 --2- 172.21.15.169:0/181706855 >> v2:172.21.15.22:3300/0 conn(0x7f2678159cd0 0x7f26780b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 -- 172.21.15.169:0/181706855 >> v2:172.21.15.22:3300/0 conn(0x7f2678159cd0 msgr2=0x7f26780b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 --2- 172.21.15.169:0/181706855 >> v2:172.21.15.22:3300/0 conn(0x7f2678159cd0 0x7f26780b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 -- 172.21.15.169:0/181706855 >> v1:172.21.15.169:6789/0 conn(0x7f267806da30 legacy=0x7f26780cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 -- 172.21.15.169:0/181706855 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780cb120 con 0x7f267806e6e0 2023-06-23T18:03:22.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f2675d9b700 1 --2- 172.21.15.169:0/181706855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f267806e6e0 0x7f267806eb60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f266c009910 tx=0x7f266c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb844378e2c6401b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267559a700 1 -- 172.21.15.169:0/181706855 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00ba20 con 0x7f267806e6e0 2023-06-23T18:03:22.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267559a700 1 -- 172.21.15.169:0/181706855 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c00bb80 con 0x7f267806e6e0 2023-06-23T18:03:22.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267559a700 1 -- 172.21.15.169:0/181706855 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c009e60 con 0x7f267806e6e0 2023-06-23T18:03:22.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- 172.21.15.169:0/181706855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f267806e6e0 msgr2=0x7f267806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 --2- 172.21.15.169:0/181706855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f267806e6e0 0x7f267806eb60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f266c009910 tx=0x7f266c009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:22.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- 172.21.15.169:0/181706855 shutdown_connections 2023-06-23T18:03:22.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 --2- 172.21.15.169:0/181706855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f267806e6e0 0x7f267806eb60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 --2- 172.21.15.169:0/181706855 >> v2:172.21.15.22:3300/0 conn(0x7f2678159cd0 0x7f26780b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- 172.21.15.169:0/181706855 >> 172.21.15.169:0/181706855 conn(0x7f267806b700 msgr2=0x7f26780b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- 172.21.15.169:0/181706855 shutdown_connections 2023-06-23T18:03:22.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 -- 172.21.15.169:0/181706855 wait complete. 2023-06-23T18:03:22.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.683+0000 7f267759e700 1 Processor -- start 2023-06-23T18:03:22.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- start start 2023-06-23T18:03:22.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 0x7f26781f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 0x7f2678153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781f2630 con 0x7f267806e6e0 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678153d80 con 0x7f2678159cd0 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780cd320 con 0x7f267806da30 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2676d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 0x7f2678153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33570/0 (socket says 172.21.15.169:33570) 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2676d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 0x7f2678153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45456/0 (socket says 172.21.15.169:45456) 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 -- 172.21.15.169:0/2541953325 learned_addr learned my addr 172.21.15.169:0/2541953325 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2675d9b700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 0x7f26781f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 -- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 msgr2=0x7f2678153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 0x7f2678153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 -- 172.21.15.169:0/2541953325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 msgr2=0x7f26781f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 0x7f26781f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 -- 172.21.15.169:0/2541953325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c009570 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2675d9b700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 0x7f26781f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267659c700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f266800d800 tx=0x7f266800db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680125f0 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2668012750 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780cd600 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668012ac0 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780cdbc0 con 0x7f267806da30 2023-06-23T18:03:22.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267804e4c0 con 0x7f267806da30 2023-06-23T18:03:22.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2668013480 con 0x7f267806da30 2023-06-23T18:03:22.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.687+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2668013d20 con 0x7f267806da30 2023-06-23T18:03:22.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.691+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2668013730 con 0x7f267806da30 2023-06-23T18:03:22.725 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:22.711+0000 7f0eb0afc2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:03:22.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.771+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05", "id": 6} v 0) v1 -- 0x7f26780c72d0 con 0x7f267806da30 2023-06-23T18:03:22.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f2666ffd700 1 -- 172.21.15.169:0/2541953325 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2668013a60 con 0x7f267806da30 2023-06-23T18:03:22.793 INFO:teuthology.orchestra.run.smithi169.stdout:6 2023-06-23T18:03:22.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 msgr2=0x7f26780ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f266800d800 tx=0x7f266800db10 comp rx=0 tx=0).stop 2023-06-23T18:03:22.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 shutdown_connections 2023-06-23T18:03:22.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2678159cd0 0x7f2678153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806e6e0 0x7f26781f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 --2- 172.21.15.169:0/2541953325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f267806da30 0x7f26780ccc90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 >> 172.21.15.169:0/2541953325 conn(0x7f267806b700 msgr2=0x7f26781e2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 shutdown_connections 2023-06-23T18:03:22.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.779+0000 7f267759e700 1 -- 172.21.15.169:0/2541953325 wait complete. 2023-06-23T18:03:22.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 1a6c4715-ddad-4f26-a63d-62554b8abbca 7 2023-06-23T18:03:22.853 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:22.839+0000 7f0eb0afc2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:03:22.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 Processor -- start 2023-06-23T18:03:22.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- start start 2023-06-23T18:03:22.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 0x7f5d0806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080cc800 con 0x7f5d080cd870 2023-06-23T18:03:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080b2c60 con 0x7f5d0806c150 2023-06-23T18:03:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0806cae0 con 0x7f5d080cc430 2023-06-23T18:03:22.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d07fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33584/0 (socket says 172.21.15.169:33584) 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d07fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45466/0 (socket says 172.21.15.169:45466) 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 -- 172.21.15.169:0/3727423596 learned_addr learned my addr 172.21.15.169:0/3727423596 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d06ffd700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 0x7f5d0806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 -- 172.21.15.169:0/3727423596 >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 msgr2=0x7f5d0806c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 -- 172.21.15.169:0/3727423596 >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 msgr2=0x7f5d0806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 0x7f5d0806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 -- 172.21.15.169:0/3727423596 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0815fa00 con 0x7f5d080cc430 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d06ffd700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 0x7f5d0806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d07fff700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d077fe700 1 --2- 172.21.15.169:0/3727423596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d080720d0 tx=0x7f5cf8009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee466f5e6b5d535d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d067fc700 1 -- 172.21.15.169:0/3727423596 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011830 con 0x7f5d080cc430 2023-06-23T18:03:22.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d067fc700 1 -- 172.21.15.169:0/3727423596 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8011990 con 0x7f5d080cc430 2023-06-23T18:03:22.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d067fc700 1 -- 172.21.15.169:0/3727423596 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011d20 con 0x7f5d080cc430 2023-06-23T18:03:22.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/3727423596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 msgr2=0x7f5d080b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/3727423596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d080720d0 tx=0x7f5cf8009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/3727423596 shutdown_connections 2023-06-23T18:03:22.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.169:3300/0 conn(0x7f5d0806c150 0x7f5d0806c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/3727423596 >> v2:172.21.15.22:3300/0 conn(0x7f5d080cd870 0x7f5d0806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/3727423596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d080b2720 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/3727423596 >> 172.21.15.169:0/3727423596 conn(0x7f5d080ccd60 msgr2=0x7f5d080cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/3727423596 shutdown_connections 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.903+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/3727423596 wait complete. 2023-06-23T18:03:22.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 Processor -- start 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 -- start start 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d080cd870 0x7f5d08148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08147ef0 con 0x7f5d080cd870 2023-06-23T18:03:22.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d081472e0 con 0x7f5d0806c150 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d0cfa6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d081475c0 con 0x7f5d080cc430 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d06ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d06ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:33592/0 (socket says 172.21.15.169:33592) 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45472/0 (socket says 172.21.15.169:45472) 2023-06-23T18:03:22.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d06ffd700 1 -- 172.21.15.169:0/1935507313 learned_addr learned my addr 172.21.15.169:0/1935507313 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d07fff700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d080cd870 0x7f5d08148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 -- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 msgr2=0x7f5d0815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 -- 172.21.15.169:0/1935507313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d080cd870 msgr2=0x7f5d08148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d080cd870 0x7f5d08148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:22.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d077fe700 1 -- 172.21.15.169:0/1935507313 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00009a70 con 0x7f5d0806c150 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.907+0000 7f5d06ffd700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5d077fe700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8016000 tx=0x7f5cf8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8004450 con 0x7f5d0806c150 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf801f070 con 0x7f5d0806c150 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf8009570 con 0x7f5d0806c150 2023-06-23T18:03:22.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8003900 con 0x7f5d0806c150 2023-06-23T18:03:22.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d08147b80 con 0x7f5d0806c150 2023-06-23T18:03:22.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5cf8003a60 con 0x7f5d0806c150 2023-06-23T18:03:22.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f5cf8021070 con 0x7f5d0806c150 2023-06-23T18:03:22.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.911+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d08147d10 con 0x7f5d0806c150 2023-06-23T18:03:22.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.915+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5cf8015020 con 0x7f5d0806c150 2023-06-23T18:03:22.990 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:22.980+0000 7f4f574e12c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:03:23.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.991+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a6c4715-ddad-4f26-a63d-62554b8abbca", "id": 7} v 0) v1 -- 0x7f5d0804e4c0 con 0x7f5d0806c150 2023-06-23T18:03:23.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5cfffff700 1 -- 172.21.15.169:0/1935507313 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a6c4715-ddad-4f26-a63d-62554b8abbca", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5cf801ae40 con 0x7f5d0806c150 2023-06-23T18:03:23.011 INFO:teuthology.orchestra.run.smithi169.stdout:7 2023-06-23T18:03:23.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 msgr2=0x7f5d081467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8016000 tx=0x7f5cf8004770 comp rx=0 tx=0).stop 2023-06-23T18:03:23.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 shutdown_connections 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d080cd870 0x7f5d08148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d080cc430 0x7f5d0815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 --2- 172.21.15.169:0/1935507313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d0806c150 0x7f5d081467e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 >> 172.21.15.169:0/1935507313 conn(0x7f5d080ccd60 msgr2=0x7f5d0815b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 shutdown_connections 2023-06-23T18:03:23.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-06-23T18:03:22.999+0000 7f5d0cfa6700 1 -- 172.21.15.169:0/1935507313 wait complete. 2023-06-23T18:03:23.025 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-23T18:03:23.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-23T18:03:23.028 INFO:tasks.ceph.osd.0:Started 2023-06-23T18:03:23.029 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-23T18:03:23.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-23T18:03:23.031 INFO:tasks.ceph.osd.1:Started 2023-06-23T18:03:23.031 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-23T18:03:23.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-23T18:03:23.034 INFO:tasks.ceph.osd.2:Started 2023-06-23T18:03:23.034 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-23T18:03:23.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-23T18:03:23.037 INFO:tasks.ceph.osd.3:Started 2023-06-23T18:03:23.037 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-23T18:03:23.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-23T18:03:23.040 INFO:tasks.ceph.osd.4:Started 2023-06-23T18:03:23.041 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-23T18:03:23.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-23T18:03:23.044 INFO:tasks.ceph.osd.5:Started 2023-06-23T18:03:23.044 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-23T18:03:23.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-23T18:03:23.047 INFO:tasks.ceph.osd.6:Started 2023-06-23T18:03:23.048 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-23T18:03:23.049 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-23T18:03:23.051 INFO:tasks.ceph.osd.7:Started 2023-06-23T18:03:23.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:23.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.152+0000 7fb8a8e15700 1 Processor -- start 2023-06-23T18:03:23.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.152+0000 7fb8a8e15700 1 -- start start 2023-06-23T18:03:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb8a40cf480 0x7fb8a40d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a40b7ec0 con 0x7fb8a40cc9e0 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40b1ad0 con 0x7fb8a40cf480 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40d0a10 con 0x7fb8a40b7af0 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb8a40cf480 0x7fb8a40d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a2ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb8a40cc9e0 0x7fb8a40ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38256/0 (socket says 172.21.15.22:38256) 2023-06-23T18:03:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a2ffd700 1 -- 172.21.15.22:0/2723029783 learned_addr learned my addr 172.21.15.22:0/2723029783 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:23.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a37fe700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:23.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2723029783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 msgr2=0x7fb8a40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:23.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a40ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2723029783 >> v1:172.21.15.22:6789/0 conn(0x7fb8a40cc9e0 legacy=0x7fb8a40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2723029783 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40d0cb0 con 0x7fb8a40cf480 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a27fc700 1 -- 172.21.15.22:0/2723029783 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592623452 0 0) 0x7fb8a40b7ec0 con 0x7fb8a40cc9e0 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a3fff700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40d18e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009d90 tx=0x7fb89c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df0f73eb631d42 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a27fc700 1 -- 172.21.15.22:0/2723029783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00e070 con 0x7fb8a40cf480 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a27fc700 1 -- 172.21.15.22:0/2723029783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c00bb20 con 0x7fb8a40cf480 2023-06-23T18:03:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a27fc700 1 -- 172.21.15.22:0/2723029783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c012430 con 0x7fb8a40cf480 2023-06-23T18:03:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2723029783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 msgr2=0x7fb8a40d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40d18e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009d90 tx=0x7fb89c00b330 comp rx=0 tx=0).stop 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2723029783 shutdown_connections 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40d18e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2723029783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2723029783 >> 172.21.15.22:0/2723029783 conn(0x7fb8a4069960 msgr2=0x7fb8a4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2723029783 shutdown_connections 2023-06-23T18:03:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.156+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2723029783 wait complete. 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 Processor -- start 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 -- start start 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a4150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a40cc9e0 0x7fb8a415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a4150e50 con 0x7fb8a40cf480 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a415d0f0 con 0x7fb8a40cc9e0 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.160+0000 7fb8a8e15700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a415e950 con 0x7fb8a40b7af0 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53822/0 (socket says 172.21.15.22:53822) 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2617198170 learned_addr learned my addr 172.21.15.22:0/2617198170 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a2ffd700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a40cc9e0 0x7fb8a415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a37fe700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 msgr2=0x7fb8a4150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a4150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a40cc9e0 msgr2=0x7fb8a415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a40cc9e0 0x7fb8a415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 -- 172.21.15.22:0/2617198170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c009a70 con 0x7fb8a40cf480 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a3fff700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8a406c750 tx=0x7fb89c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c012430 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c020070 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a415c200 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c004280 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a415c7c0 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb89c0046e0 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb89c022070 con 0x7fb8a40cf480 2023-06-23T18:03:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.164+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a404e4c0 con 0x7fb8a40cf480 2023-06-23T18:03:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.168+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fb89c012e20 con 0x7fb8a40cf480 2023-06-23T18:03:23.194 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:23.179+0000 7f0eb0afc2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:03:23.195 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:23.184+0000 7f4f574e12c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:03:23.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.232+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8a415d230 con 0x7fb8a40cf480 2023-06-23T18:03:23.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb88bfff700 1 -- 172.21.15.22:0/2617198170 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb89c027480 con 0x7fb8a40cf480 2023-06-23T18:03:23.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:23.245 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:22.997726+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:23.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 msgr2=0x7fb8a40b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:23.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8a406c750 tx=0x7fb89c004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 shutdown_connections 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8a40cf480 0x7fb8a40b3600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a40cc9e0 0x7fb8a415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 --2- 172.21.15.22:0/2617198170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a40b7af0 0x7fb8a4150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 >> 172.21.15.22:0/2617198170 conn(0x7fb8a4069960 msgr2=0x7fb8a415b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 shutdown_connections 2023-06-23T18:03:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:23.236+0000 7fb8a8e15700 1 -- 172.21.15.22:0/2617198170 wait complete. 2023-06-23T18:03:23.257 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-23T18:03:23.258 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-23T18:03:23.272 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:23.260+0000 7f4f574e12c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:03:23.331 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:23.320+0000 7f4f574e12c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:03:23.361 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:23.347+0000 7f0eb0afc2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:03:23.398 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:03:23.388+0000 7f98b7e6b700 -1 Falling back to public interface 2023-06-23T18:03:23.411 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:03:23.399+0000 7fa2f54e3700 -1 Falling back to public interface 2023-06-23T18:03:23.412 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:03:23.400+0000 7efd57df5700 -1 Falling back to public interface 2023-06-23T18:03:23.413 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:03:23.399+0000 7f777045f700 -1 Falling back to public interface 2023-06-23T18:03:23.413 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:03:23.400+0000 7fcb6119c700 -1 Falling back to public interface 2023-06-23T18:03:23.413 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:03:23.400+0000 7fcb60167700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-06-23T18:03:23.428 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:03:23.415+0000 7eff80ffc700 -1 Falling back to public interface 2023-06-23T18:03:23.433 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:23.419+0000 7f0eb0afc2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:03:23.487 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:23.475+0000 7f0eb0afc2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:03:23.664 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:03:23.651+0000 7fb0fcebc700 -1 Falling back to public interface 2023-06-23T18:03:23.679 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:03:23.668+0000 7fccafc3b700 -1 Falling back to public interface 2023-06-23T18:03:24.065 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:24.144 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.132+0000 7f4f574e12c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:03:24.172 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 Processor -- start 2023-06-23T18:03:24.172 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 -- start start 2023-06-23T18:03:24.173 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f405006d090 0x7f40500cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:24.173 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f405006d4c0 0x7f40500cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:24.173 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500ab8f0 con 0x7f40500cdf10 2023-06-23T18:03:24.173 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4050065990 con 0x7f405006d4c0 2023-06-23T18:03:24.174 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.160+0000 7f4058407700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405006fe50 con 0x7f405006d090 2023-06-23T18:03:24.174 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f405006d4c0 0x7f40500cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:24.174 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40559a2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f405006d090 0x7f40500cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:24.174 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f405006d4c0 0x7f40500cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:37124/0 (socket says 172.21.15.22:37124) 2023-06-23T18:03:24.174 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 -- 172.21.15.22:0/3622986085 learned_addr learned my addr 172.21.15.22:0/3622986085 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:24.175 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 -- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f405006d090 msgr2=0x7f40500cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:24.175 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 --2- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f405006d090 0x7f40500cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 -- 172.21.15.22:0/3622986085 >> v1:172.21.15.169:6789/0 conn(0x7f40500cdf10 legacy=0x7f40500ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 -- 172.21.15.22:0/3622986085 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050070130 con 0x7f405006d4c0 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40569a4700 1 --2- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d4c0 0x7f40500cb6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4048009e50 tx=0x7f404800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57d7c75ecf9c796 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40551a1700 1 -- 172.21.15.22:0/3622986085 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800e070 con 0x7f405006d4c0 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40551a1700 1 -- 172.21.15.22:0/3622986085 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404800b910 con 0x7f405006d4c0 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f40551a1700 1 -- 172.21.15.22:0/3622986085 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800bcc0 con 0x7f405006d4c0 2023-06-23T18:03:24.176 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 -- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d4c0 msgr2=0x7f40500cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 --2- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d4c0 0x7f40500cb6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4048009e50 tx=0x7f404800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 -- 172.21.15.22:0/3622986085 shutdown_connections 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 --2- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d4c0 0x7f40500cb6c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 --2- 172.21.15.22:0/3622986085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f405006d090 0x7f40500cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 -- 172.21.15.22:0/3622986085 >> 172.21.15.22:0/3622986085 conn(0x7f4050071c20 msgr2=0x7f405006a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 -- 172.21.15.22:0/3622986085 shutdown_connections 2023-06-23T18:03:24.177 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.164+0000 7f4058407700 1 -- 172.21.15.22:0/3622986085 wait complete. 2023-06-23T18:03:24.178 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 Processor -- start 2023-06-23T18:03:24.178 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 -- start start 2023-06-23T18:03:24.178 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d090 0x7f40500cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:24.179 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:24.179 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40500cdf10 0x7f405014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:24.179 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500cb910 con 0x7f405006d4c0 2023-06-23T18:03:24.179 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500b4850 con 0x7f405006d090 2023-06-23T18:03:24.179 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4050150500 con 0x7f40500cdf10 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53828/0 (socket says 172.21.15.22:53828) 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 -- 172.21.15.22:0/1043832811 learned_addr learned my addr 172.21.15.22:0/1043832811 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40561a3700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d090 0x7f40500cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40569a4700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40500cdf10 0x7f405014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:24.181 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 -- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40500cdf10 msgr2=0x7f405014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40500cdf10 0x7f405014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 -- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d090 msgr2=0x7f40500cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d090 0x7f40500cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 -- 172.21.15.22:0/1043832811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048009b30 con 0x7f405006d4c0 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f40559a2700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f404000ea00 tx=0x7f404000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040009720 con 0x7f405006d4c0 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404000eea0 con 0x7f405006d4c0 2023-06-23T18:03:24.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500cac90 con 0x7f405006d4c0 2023-06-23T18:03:24.183 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.168+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000ca50 con 0x7f405006d4c0 2023-06-23T18:03:24.183 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.172+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40500cb250 con 0x7f405006d4c0 2023-06-23T18:03:24.183 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.172+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4040005a40 con 0x7f405006d4c0 2023-06-23T18:03:24.183 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.172+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4040006750 con 0x7f405006d4c0 2023-06-23T18:03:24.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.176+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405004e4c0 con 0x7f405006d4c0 2023-06-23T18:03:24.188 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.176+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4040021060 con 0x7f405006d4c0 2023-06-23T18:03:24.211 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.200+0000 7f4f574e12c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:03:24.253 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.240+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f405006fa60 con 0x7f405006d4c0 2023-06-23T18:03:24.254 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4046ffd700 1 -- 172.21.15.22:0/1043832811 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4040017070 con 0x7f405006d4c0 2023-06-23T18:03:24.254 INFO:teuthology.misc.health.smithi022.stdout: 2023-06-23T18:03:24.254 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":10,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:22.997726+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:24.255 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 msgr2=0x7f40500c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f404000ea00 tx=0x7f404000ed10 comp rx=0 tx=0).stop 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 shutdown_connections 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40500cdf10 0x7f405014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f405006d4c0 0x7f40500c5060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 --2- 172.21.15.22:0/1043832811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f405006d090 0x7f40500cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:24.256 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 >> 172.21.15.22:0/1043832811 conn(0x7f4050071c20 msgr2=0x7f405015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:24.257 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 shutdown_connections 2023-06-23T18:03:24.257 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:24.244+0000 7f4058407700 1 -- 172.21.15.22:0/1043832811 wait complete. 2023-06-23T18:03:24.266 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-23T18:03:24.301 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.287+0000 7f0eb0afc2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:03:24.349 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.340+0000 7f4f574e12c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:03:24.368 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.355+0000 7f0eb0afc2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:03:24.416 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.404+0000 7f4f574e12c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:03:24.492 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.480+0000 7f4f574e12c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:03:24.521 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.507+0000 7f0eb0afc2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:03:24.585 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.571+0000 7f0eb0afc2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:03:24.654 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.643+0000 7f0eb0afc2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:03:24.673 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.663+0000 7f4f574e12c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:03:24.815 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:24.803+0000 7f0eb0afc2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:03:24.899 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:24.887+0000 7f4f574e12c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:03:25.018 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:25.003+0000 7f0eb0afc2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:03:25.573 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:25.563+0000 7f4f574e12c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:03:25.664 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-06-23T18:03:25.651+0000 7f0eb0afc2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:03:26.623 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:26.611+0000 7f4f3e766700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-23T18:03:26.628 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-06-23T18:03:26.615+0000 7f4f3e766700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-23T18:03:26.847 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:03:26.835+0000 7efd57df5700 -1 osd.0 0 log_to_monitors true 2023-06-23T18:03:27.021 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:03:27.007+0000 7fa2f54e3700 -1 osd.7 0 log_to_monitors true 2023-06-23T18:03:27.022 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:03:27.011+0000 7eff80ffc700 -1 osd.5 0 log_to_monitors true 2023-06-23T18:03:27.124 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:03:27.111+0000 7fccafc3b700 -1 osd.3 0 log_to_monitors true 2023-06-23T18:03:27.218 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:03:27.203+0000 7f777045f700 -1 osd.4 0 log_to_monitors true 2023-06-23T18:03:27.293 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:03:27.279+0000 7f98b7e6b700 -1 osd.2 0 log_to_monitors true 2023-06-23T18:03:27.342 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:03:27.331+0000 7fcb6119c700 -1 osd.1 0 log_to_monitors true 2023-06-23T18:03:27.505 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:03:27.491+0000 7fb0fcebc700 -1 osd.6 0 log_to_monitors true 2023-06-23T18:03:30.219 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:03:30.207+0000 7fcb5cfdb700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T18:03:30.226 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:03:30.211+0000 7efd53c34700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T18:03:30.231 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:03:30.219+0000 7fcb54a8a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.231 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:03:30.219+0000 7efd4b6e3700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.249 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:03:30.235+0000 7fccaba7a700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T18:03:30.254 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:03:30.243+0000 7fcca3529700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.259 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:03:30.247+0000 7f98b3caa700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T18:03:30.265 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:03:30.251+0000 7f98ab759700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.856 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:03:30.843+0000 7fa2f1ca9700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T18:03:30.863 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:03:30.851+0000 7fa2e8dd1700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.938 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:03:30.923+0000 7fb0f8cfb700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T18:03:30.947 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:03:30.935+0000 7fb0f07aa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:30.967 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:03:30.955+0000 7eff7d7c2700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T18:03:30.977 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:03:30.963+0000 7eff748ea700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:31.023 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:03:31.011+0000 7f776cc25700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T18:03:31.031 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:03:31.019+0000 7f7763d4d700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:03:31.072 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:31.182 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 Processor -- start 2023-06-23T18:03:31.183 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- start start 2023-06-23T18:03:31.184 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe92815c7c0 0x7fe9280acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280afb80 0x7fe9280affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280cbc10 con 0x7fe9280cd700 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280b04f0 con 0x7fe92815c7c0 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280cbe60 con 0x7fe9280afb80 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe92815c7c0 0x7fe9280acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.185 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe92815c7c0 0x7fe9280acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49416/0 (socket says 172.21.15.22:49416) 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 -- 172.21.15.22:0/1403608951 learned_addr learned my addr 172.21.15.22:0/1403608951 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe926d9d700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280afb80 0x7fe9280affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 -- 172.21.15.22:0/1403608951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280afb80 msgr2=0x7fe9280affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280afb80 0x7fe9280affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 -- 172.21.15.22:0/1403608951 >> v1:172.21.15.169:6789/0 conn(0x7fe9280cd700 legacy=0x7fe92815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 -- 172.21.15.22:0/1403608951 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280cc140 con 0x7fe92815c7c0 2023-06-23T18:03:31.186 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92659c700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe92815c7c0 0x7fe9280acb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009910 tx=0x7fe91c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3111f2545e69a55 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.187 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92559a700 1 -- 172.21.15.22:0/1403608951 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011830 con 0x7fe92815c7c0 2023-06-23T18:03:31.187 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92559a700 1 -- 172.21.15.22:0/1403608951 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c011990 con 0x7fe92815c7c0 2023-06-23T18:03:31.187 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92559a700 1 -- 172.21.15.22:0/1403608951 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011d40 con 0x7fe92815c7c0 2023-06-23T18:03:31.188 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- 172.21.15.22:0/1403608951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe92815c7c0 msgr2=0x7fe9280acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.188 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe92815c7c0 0x7fe9280acb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009910 tx=0x7fe91c009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:31.188 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- 172.21.15.22:0/1403608951 shutdown_connections 2023-06-23T18:03:31.188 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280afb80 0x7fe9280affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 --2- 172.21.15.22:0/1403608951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe92815c7c0 0x7fe9280acb10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- 172.21.15.22:0/1403608951 >> 172.21.15.22:0/1403608951 conn(0x7fe92806e870 msgr2=0x7fe92806ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- 172.21.15.22:0/1403608951 shutdown_connections 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.170+0000 7fe92759e700 1 -- 172.21.15.22:0/1403608951 wait complete. 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 Processor -- start 2023-06-23T18:03:31.189 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 -- start start 2023-06-23T18:03:31.191 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.191 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280cd700 0x7fe9280c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.191 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92815c7c0 0x7fe9280ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.191 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281f1bb0 con 0x7fe9280afb80 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281f5ca0 con 0x7fe92815c7c0 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281f59c0 con 0x7fe9280cd700 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49424/0 (socket says 172.21.15.22:49424) 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 -- 172.21.15.22:0/788019886 learned_addr learned my addr 172.21.15.22:0/788019886 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe925d9b700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280cd700 0x7fe9280c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.192 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe926d9d700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92815c7c0 0x7fe9280ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 -- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280cd700 msgr2=0x7fe9280c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280cd700 0x7fe9280c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 -- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92815c7c0 msgr2=0x7fe9280ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92815c7c0 0x7fe9280ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.174+0000 7fe92659c700 1 -- 172.21.15.22:0/788019886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c009570 con 0x7fe9280afb80 2023-06-23T18:03:31.193 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe92659c700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.194 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c004610 con 0x7fe9280afb80 2023-06-23T18:03:31.194 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c0036c0 con 0x7fe9280afb80 2023-06-23T18:03:31.194 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9280c5240 con 0x7fe9280afb80 2023-06-23T18:03:31.194 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003a50 con 0x7fe9280afb80 2023-06-23T18:03:31.194 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9280c5800 con 0x7fe9280afb80 2023-06-23T18:03:31.195 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92804e4c0 con 0x7fe9280afb80 2023-06-23T18:03:31.195 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7fe91c003bb0 con 0x7fe9280afb80 2023-06-23T18:03:31.195 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 0x7fe908068710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.196 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe91c021030 con 0x7fe9280afb80 2023-06-23T18:03:31.196 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe925d9b700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 0x7fe908068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.196 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.178+0000 7fe925d9b700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 0x7fe908068710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9280ca130 tx=0x7fe918009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.200 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.186+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe91c052910 con 0x7fe9280afb80 2023-06-23T18:03:31.371 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.358+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9280b3f20 con 0x7fe9280afb80 2023-06-23T18:03:31.373 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.358+0000 7fe916ffd700 1 -- 172.21.15.22:0/788019886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7fe91c003e60 con 0x7fe9280afb80 2023-06-23T18:03:31.373 INFO:teuthology.misc.health.smithi022.stdout: 2023-06-23T18:03:31.373 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":13,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:31.218404+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:31.374 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 msgr2=0x7fe908068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.375 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 0x7fe908068710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9280ca130 tx=0x7fe918009040 comp rx=0 tx=0).stop 2023-06-23T18:03:31.375 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 msgr2=0x7fe928152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.375 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004770 comp rx=0 tx=0).stop 2023-06-23T18:03:31.377 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 shutdown_connections 2023-06-23T18:03:31.377 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe908066260 0x7fe908068710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92815c7c0 0x7fe9280ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9280cd700 0x7fe9280c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 --2- 172.21.15.22:0/788019886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9280afb80 0x7fe928152500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 >> 172.21.15.22:0/788019886 conn(0x7fe92806e870 msgr2=0x7fe92806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 shutdown_connections 2023-06-23T18:03:31.378 INFO:teuthology.misc.health.smithi022.stderr:2023-06-23T18:03:31.362+0000 7fe92759e700 1 -- 172.21.15.22:0/788019886 wait complete. 2023-06-23T18:03:31.385 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-23T18:03:31.386 INFO:tasks.ceph:Creating RBD pool 2023-06-23T18:03:31.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-23T18:03:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.470+0000 7f9871bc5700 1 Processor -- start 2023-06-23T18:03:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.470+0000 7f9871bc5700 1 -- start start 2023-06-23T18:03:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9871bc5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9871bc5700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c15bc30 con 0x7f986c0cc3d0 2023-06-23T18:03:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9871bc5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c070c40 con 0x7f986c0cc820 2023-06-23T18:03:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9871bc5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c071500 con 0x7f986c155c20 2023-06-23T18:03:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f986c0cc3d0 0x7f986c06fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52820/0 (socket says 172.21.15.22:52820) 2023-06-23T18:03:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986bfff700 1 -- 172.21.15.22:0/1942687867 learned_addr learned my addr 172.21.15.22:0/1942687867 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9870bc3700 1 --2- 172.21.15.22:0/1942687867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332020161 0 0) 0x7f986c15bc30 con 0x7f986c0cc3d0 2023-06-23T18:03:31.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985c0046e0 con 0x7f986c0cc3d0 2023-06-23T18:03:31.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558241447 0 0) 0x7f986c070c40 con 0x7f986c0cc820 2023-06-23T18:03:31.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f986c15bc30 con 0x7f986c0cc820 2023-06-23T18:03:31.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9870bc3700 1 -- 172.21.15.22:0/1942687867 >> v1:172.21.15.169:6789/0 conn(0x7f986c0cc820 legacy=0x7f986c070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9870bc3700 1 -- 172.21.15.22:0/1942687867 >> v1:172.21.15.22:6789/0 conn(0x7f986c0cc3d0 legacy=0x7f986c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9870bc3700 1 -- 172.21.15.22:0/1942687867 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c15be80 con 0x7f986c155c20 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f9870bc3700 1 --2- 172.21.15.22:0/1942687867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c0b0da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9860009880 tx=0x7f9860009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2945c9cea80b12 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860011830 con 0x7f986c155c20 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9860011990 con 0x7f986c155c20 2023-06-23T18:03:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.474+0000 7f986b7fe700 1 -- 172.21.15.22:0/1942687867 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860011d20 con 0x7f986c155c20 2023-06-23T18:03:31.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 -- 172.21.15.22:0/1942687867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 msgr2=0x7f986c0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 --2- 172.21.15.22:0/1942687867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c0b0da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9860009880 tx=0x7f9860009b90 comp rx=0 tx=0).stop 2023-06-23T18:03:31.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 -- 172.21.15.22:0/1942687867 shutdown_connections 2023-06-23T18:03:31.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 --2- 172.21.15.22:0/1942687867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c0b0da0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 -- 172.21.15.22:0/1942687867 >> 172.21.15.22:0/1942687867 conn(0x7f986c06ca60 msgr2=0x7f986c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:31.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 -- 172.21.15.22:0/1942687867 shutdown_connections 2023-06-23T18:03:31.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.478+0000 7f9871bc5700 1 -- 172.21.15.22:0/1942687867 wait complete. 2023-06-23T18:03:31.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 Processor -- start 2023-06-23T18:03:31.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- start start 2023-06-23T18:03:31.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986c0cc820 0x7f986c0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c14ed20 con 0x7f986c14e340 2023-06-23T18:03:31.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c151c10 con 0x7f986c0cc820 2023-06-23T18:03:31.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1f3160 con 0x7f986c155c20 2023-06-23T18:03:31.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49434/0 (socket says 172.21.15.22:49434) 2023-06-23T18:03:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 -- 172.21.15.22:0/3154501097 learned_addr learned my addr 172.21.15.22:0/3154501097 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9870bc3700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986c0cc820 0x7f986c0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f986bfff700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 -- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 msgr2=0x7f986c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 -- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986c0cc820 msgr2=0x7f986c0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986c0cc820 0x7f986c0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 -- 172.21.15.22:0/3154501097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600094b0 con 0x7f986c14e340 2023-06-23T18:03:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98713c4700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f986400cbc0 tx=0x7f986400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003c20 con 0x7f986c14e340 2023-06-23T18:03:31.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864003d80 con 0x7f986c14e340 2023-06-23T18:03:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c151e60 con 0x7f986c14e340 2023-06-23T18:03:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400aaa0 con 0x7f986c14e340 2023-06-23T18:03:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.482+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c152420 con 0x7f986c14e340 2023-06-23T18:03:31.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c04e4c0 con 0x7f986c14e340 2023-06-23T18:03:31.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7f986400ac00 con 0x7f986c14e340 2023-06-23T18:03:31.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f98697fa700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 0x7f9850068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:31.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9864083530 con 0x7f986c14e340 2023-06-23T18:03:31.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f9870bc3700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 0x7f9850068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:31.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.486+0000 7f9870bc3700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 0x7f9850068740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f98600090b0 tx=0x7f986001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:31.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.498+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f98640510b0 con 0x7f986c14e340 2023-06-23T18:03:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:31.654+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f986c0cf460 con 0x7f986c14e340 2023-06-23T18:03:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.226+0000 7f98697fa700 1 -- 172.21.15.22:0/3154501097 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f98640546d0 con 0x7f986c14e340 2023-06-23T18:03:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 msgr2=0x7f9850068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 0x7f9850068740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f98600090b0 tx=0x7f986001c040 comp rx=0 tx=0).stop 2023-06-23T18:03:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 msgr2=0x7f986c14e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f986400cbc0 tx=0x7f986400ced0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 shutdown_connections 2023-06-23T18:03:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f986c14e340 0x7f986c14e7e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f986c155c20 0x7f986c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9850066290 0x7f9850068740 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 --2- 172.21.15.22:0/3154501097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986c0cc820 0x7f986c0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 >> 172.21.15.22:0/3154501097 conn(0x7f986c0cc3d0 msgr2=0x7f986c14f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 shutdown_connections 2023-06-23T18:03:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.230+0000 7f9871bc5700 1 -- 172.21.15.22:0/3154501097 wait complete. 2023-06-23T18:03:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'rbd' created 2023-06-23T18:03:32.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-23T18:03:32.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 Processor -- start 2023-06-23T18:03:32.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 -- start start 2023-06-23T18:03:32.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e780d1f30 0x7f6e780d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 0x7f6e780d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780ccb30 con 0x7f6e780cf3a0 2023-06-23T18:03:32.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780d2880 con 0x7f6e780d1f30 2023-06-23T18:03:32.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7d1a9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780d2b60 con 0x7f6e780d2360 2023-06-23T18:03:32.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e777fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e780d1f30 0x7f6e780d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.362+0000 7f6e7c9a8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 0x7f6e780d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e780d1f30 0x7f6e780d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:49714/0 (socket says 172.21.15.22:49714) 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 -- 172.21.15.22:0/2944360688 learned_addr learned my addr 172.21.15.22:0/2944360688 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 -- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 msgr2=0x7f6e780d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 0x7f6e780d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 -- 172.21.15.22:0/2944360688 >> v1:172.21.15.169:6789/0 conn(0x7f6e780cf3a0 legacy=0x7f6e780cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 -- 172.21.15.22:0/2944360688 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780d2e40 con 0x7f6e780d1f30 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7c9a8700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 0x7f6e780d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:32.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d1f30 0x7f6e780d0020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009910 tx=0x7f6e60009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ee99a1d5966032a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e76ffd700 1 -- 172.21.15.22:0/2944360688 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60011830 con 0x7f6e780d1f30 2023-06-23T18:03:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e76ffd700 1 -- 172.21.15.22:0/2944360688 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60011990 con 0x7f6e780d1f30 2023-06-23T18:03:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e76ffd700 1 -- 172.21.15.22:0/2944360688 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60011d40 con 0x7f6e780d1f30 2023-06-23T18:03:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d1f30 msgr2=0x7f6e780d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d1f30 0x7f6e780d0020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009910 tx=0x7f6e60009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2944360688 shutdown_connections 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d2360 0x7f6e780d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2944360688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d1f30 0x7f6e780d0020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2944360688 >> 172.21.15.22:0/2944360688 conn(0x7f6e7806a280 msgr2=0x7f6e7809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2944360688 shutdown_connections 2023-06-23T18:03:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.366+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2944360688 wait complete. 2023-06-23T18:03:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 Processor -- start 2023-06-23T18:03:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- start start 2023-06-23T18:03:32.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 0x7f6e780c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d2360 0x7f6e780c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780afc40 con 0x7f6e780cf3a0 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780aff20 con 0x7f6e780d2360 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780b0200 con 0x7f6e780d1f30 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e77fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e77fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49444/0 (socket says 172.21.15.22:49444) 2023-06-23T18:03:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e77fff700 1 -- 172.21.15.22:0/2202013528 learned_addr learned my addr 172.21.15.22:0/2202013528 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 0x7f6e780c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7c9a8700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d2360 0x7f6e780c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 -- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d2360 msgr2=0x7f6e780c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d2360 0x7f6e780c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 -- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 msgr2=0x7f6e780b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 -- 172.21.15.22:0/2202013528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70009a70 con 0x7f6e780d1f30 2023-06-23T18:03:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e77fff700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e777fe700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 0x7f6e780c7ee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e60016000 tx=0x7f6e60004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600044d0 con 0x7f6e780d1f30 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e60009570 con 0x7f6e780d1f30 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e600036c0 con 0x7f6e780d1f30 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60003a70 con 0x7f6e780d1f30 2023-06-23T18:03:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.370+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e780b07c0 con 0x7f6e780d1f30 2023-06-23T18:03:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7804e4c0 con 0x7f6e780d1f30 2023-06-23T18:03:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7f6e60003bd0 con 0x7f6e780d1f30 2023-06-23T18:03:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e74ff9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 0x7f6e58068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f6e60021030 con 0x7f6e780d1f30 2023-06-23T18:03:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e77fff700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 0x7f6e58068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.374+0000 7f6e77fff700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 0x7f6e58068740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007fd0 tx=0x7f6e68009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.386+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6e600526e0 con 0x7f6e780d1f30 2023-06-23T18:03:32.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:32.550+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6e78144f30 con 0x7f6e780d1f30 2023-06-23T18:03:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.234+0000 7f6e74ff9700 1 -- 172.21.15.22:0/2202013528 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6e60055d00 con 0x7f6e780d1f30 2023-06-23T18:03:33.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 msgr2=0x7f6e58068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 0x7f6e58068740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007fd0 tx=0x7f6e68009040 comp rx=0 tx=0).stop 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 msgr2=0x7f6e780c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 0x7f6e780c7ee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e60016000 tx=0x7f6e60004770 comp rx=0 tx=0).stop 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 shutdown_connections 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6e58066290 0x7f6e58068740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e780d2360 0x7f6e780c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e780d1f30 0x7f6e780c7ee0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 --2- 172.21.15.22:0/2202013528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e780cf3a0 0x7f6e780b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 >> 172.21.15.22:0/2202013528 conn(0x7f6e7806a280 msgr2=0x7f6e780cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 shutdown_connections 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.246+0000 7f6e7d1a9700 1 -- 172.21.15.22:0/2202013528 wait complete. 2023-06-23T18:03:33.261 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-23T18:03:33.272 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-23T18:03:33.272 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T18:03:33.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T18:03:33.275 INFO:tasks.ceph.mds.a:Started 2023-06-23T18:03:33.275 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T18:03:33.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T18:03:33.279 INFO:tasks.ceph.mds.c:Started 2023-06-23T18:03:33.279 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T18:03:33.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T18:03:33.281 INFO:tasks.ceph.mds.b:Started 2023-06-23T18:03:33.282 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T18:03:33.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T18:03:33.284 INFO:tasks.ceph.mds.d:Started 2023-06-23T18:03:33.285 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-23T18:03:33.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:33.358 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2023-06-23T18:03:33.359 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2023-06-23T18:03:33.363 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-06-23T18:03:33.368 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-06-23T18:03:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.394+0000 7f1f0a3f6700 1 Processor -- start 2023-06-23T18:03:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.394+0000 7f1f0a3f6700 1 -- start start 2023-06-23T18:03:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f040cd450 0x7f1f04159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cabd0 0x7f1f040cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0415cd20 con 0x7f1f0415c5d0 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f040cb590 con 0x7f1f040cd450 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0415d000 con 0x7f1f040cabd0 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f040cd450 0x7f1f04159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f08993700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cabd0 0x7f1f040cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f040cd450 0x7f1f04159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49486/0 (socket says 172.21.15.22:49486) 2023-06-23T18:03:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 -- 172.21.15.22:0/2138586383 learned_addr learned my addr 172.21.15.22:0/2138586383 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 -- 172.21.15.22:0/2138586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cabd0 msgr2=0x7f1f040cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 --2- 172.21.15.22:0/2138586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cabd0 0x7f1f040cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 -- 172.21.15.22:0/2138586383 >> v1:172.21.15.169:6789/0 conn(0x7f1f0415c5d0 legacy=0x7f1f040cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f02ffd700 1 -- 172.21.15.22:0/2138586383 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711414788 0 0) 0x7f1f0415cd20 con 0x7f1f0415c5d0 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 -- 172.21.15.22:0/2138586383 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0415d2e0 con 0x7f1f040cd450 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f03fff700 1 --2- 172.21.15.22:0/2138586383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f040cd450 0x7f1f04159aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009910 tx=0x7f1ef8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7b2d2ccfba89d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f02ffd700 1 -- 172.21.15.22:0/2138586383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011830 con 0x7f1f040cd450 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f02ffd700 1 -- 172.21.15.22:0/2138586383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8011990 con 0x7f1f040cd450 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f02ffd700 1 -- 172.21.15.22:0/2138586383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011d40 con 0x7f1f040cd450 2023-06-23T18:03:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/2138586383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f040cd450 msgr2=0x7f1f04159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/2138586383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f040cd450 0x7f1f04159aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009910 tx=0x7f1ef8009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/2138586383 shutdown_connections 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/2138586383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cabd0 0x7f1f040cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/2138586383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f040cd450 0x7f1f04159aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/2138586383 >> 172.21.15.22:0/2138586383 conn(0x7f1f04069960 msgr2=0x7f1f04069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/2138586383 shutdown_connections 2023-06-23T18:03:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.398+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/2138586383 wait complete. 2023-06-23T18:03:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 Processor -- start 2023-06-23T18:03:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 -- start start 2023-06-23T18:03:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 0x7f1f040ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 0x7f1f040c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041505e0 con 0x7f1f0415c5d0 2023-06-23T18:03:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041507d0 con 0x7f1f040cabd0 2023-06-23T18:03:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f0a3f6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04150ab0 con 0x7f1f040cd450 2023-06-23T18:03:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 0x7f1f040ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.402+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 0x7f1f040ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:49768/0 (socket says 172.21.15.22:49768) 2023-06-23T18:03:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:54176/0 (socket says 172.21.15.22:54176) 2023-06-23T18:03:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f03fff700 1 -- 172.21.15.22:0/1507124999 learned_addr learned my addr 172.21.15.22:0/1507124999 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f08993700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 0x7f1f040c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 -- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 msgr2=0x7f1f040ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 0x7f1f040ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 -- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 msgr2=0x7f1f040c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 0x7f1f040c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 -- 172.21.15.22:0/1507124999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8009570 con 0x7f1f040cd450 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f08993700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 0x7f1f040c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f037fe700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00ef80 tx=0x7f1efc010330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc013070 con 0x7f1f040cd450 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00e2e0 con 0x7f1f040cd450 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f04150d90 con 0x7f1f040cd450 2023-06-23T18:03:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc003c60 con 0x7f1f040cd450 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f04151570 con 0x7f1f040cd450 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0404dc70 con 0x7f1f040cd450 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7f1efc010a30 con 0x7f1f040cd450 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 0x7f1ef0068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1efc0875e0 con 0x7f1f040cd450 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.406+0000 7f1f03fff700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 0x7f1ef0068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.410+0000 7f1f03fff700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 0x7f1ef0068740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009c60 tx=0x7f1ef8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.414+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1efc054e90 con 0x7f1f040cd450 2023-06-23T18:03:33.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.582+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f040c65c0 con 0x7f1f040cd450 2023-06-23T18:03:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f00ff9700 1 -- 172.21.15.22:0/1507124999 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14650 (secure 0 0 0) 0x7f1efc0584b0 con 0x7f1f040cd450 2023-06-23T18:03:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:33.600 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":15,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:33.225543+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:33.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 msgr2=0x7f1ef0068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 0x7f1ef0068740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009c60 tx=0x7f1ef8008030 comp rx=0 tx=0).stop 2023-06-23T18:03:33.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 msgr2=0x7f1f040aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00ef80 tx=0x7f1efc010330 comp rx=0 tx=0).stop 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 shutdown_connections 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f0415c5d0 0x7f1f040c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f040cd450 0x7f1f040aeb90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef0066290 0x7f1ef0068740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 --2- 172.21.15.22:0/1507124999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040cabd0 0x7f1f040ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 >> 172.21.15.22:0/1507124999 conn(0x7f1f04069960 msgr2=0x7f1f040c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:33.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 shutdown_connections 2023-06-23T18:03:33.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.586+0000 7f1f0a3f6700 1 -- 172.21.15.22:0/1507124999 wait complete. 2023-06-23T18:03:33.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:03:33.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:03:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 Processor -- start 2023-06-23T18:03:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 -- start start 2023-06-23T18:03:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd96015be20 0x7fd96006aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 0x7fd960069960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600b0e80 con 0x7fd96015be20 2023-06-23T18:03:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600ce6e0 con 0x7fd9600b0a20 2023-06-23T18:03:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd964952700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006af50 con 0x7fd9600cefa0 2023-06-23T18:03:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd95f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd96015be20 0x7fd96006aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd95f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd96015be20 0x7fd96006aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49516/0 (socket says 172.21.15.22:49516) 2023-06-23T18:03:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd95f7fe700 1 -- 172.21.15.22:0/1989922251 learned_addr learned my addr 172.21.15.22:0/1989922251 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd95ffff700 1 --2- 172.21.15.22:0/1989922251 >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 0x7fd960069960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.706+0000 7fd95effd700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 -- 172.21.15.22:0/1989922251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 msgr2=0x7fd9600b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 -- 172.21.15.22:0/1989922251 >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 msgr2=0x7fd960069960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/1989922251 >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 0x7fd960069960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 -- 172.21.15.22:0/1989922251 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96015f900 con 0x7fd96015be20 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95ffff700 1 --2- 172.21.15.22:0/1989922251 >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 0x7fd960069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd96015be20 0x7fd96006aa10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd960073320 tx=0x7fd950009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b18dd0ece4f5341 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95e7fc700 1 -- 172.21.15.22:0/1989922251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950011830 con 0x7fd96015be20 2023-06-23T18:03:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95e7fc700 1 -- 172.21.15.22:0/1989922251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950011990 con 0x7fd96015be20 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95e7fc700 1 -- 172.21.15.22:0/1989922251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950011d20 con 0x7fd96015be20 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- 172.21.15.22:0/1989922251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd96015be20 msgr2=0x7fd96006aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd96015be20 0x7fd96006aa10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd960073320 tx=0x7fd950009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- 172.21.15.22:0/1989922251 shutdown_connections 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- 172.21.15.22:0/1989922251 >> v2:172.21.15.169:3300/0 conn(0x7fd9600b0a20 0x7fd960069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- 172.21.15.22:0/1989922251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd96015be20 0x7fd96006aa10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- 172.21.15.22:0/1989922251 >> 172.21.15.22:0/1989922251 conn(0x7fd96006ca60 msgr2=0x7fd9600ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- 172.21.15.22:0/1989922251 shutdown_connections 2023-06-23T18:03:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- 172.21.15.22:0/1989922251 wait complete. 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 Processor -- start 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- start start 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 0x7fd960156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601574e0 con 0x7fd960155d80 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601561f0 con 0x7fd960156b30 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd964952700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f25e0 con 0x7fd9600cefa0 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.710+0000 7fd95effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49532/0 (socket says 172.21.15.22:49532) 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 -- 172.21.15.22:0/211978405 learned_addr learned my addr 172.21.15.22:0/211978405 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95ffff700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 0x7fd960156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 -- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 msgr2=0x7fd9600b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 -- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 msgr2=0x7fd960156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 0x7fd960156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 -- 172.21.15.22:0/211978405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950009570 con 0x7fd960155d80 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95ffff700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 0x7fd960156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd95effd700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e8a0 tx=0x7fd94800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9480095c0 con 0x7fd960155d80 2023-06-23T18:03:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd948013680 con 0x7fd960155d80 2023-06-23T18:03:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9600b31e0 con 0x7fd960155d80 2023-06-23T18:03:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948013a10 con 0x7fd960155d80 2023-06-23T18:03:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9600b37a0 con 0x7fd960155d80 2023-06-23T18:03:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.714+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96004e4c0 con 0x7fd960155d80 2023-06-23T18:03:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.718+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7fd948013b70 con 0x7fd960155d80 2023-06-23T18:03:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.718+0000 7fd957fff700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 0x7fd940068710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.718+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd948083810 con 0x7fd960155d80 2023-06-23T18:03:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.718+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 0x7fd940068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.718+0000 7fd95f7fe700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 0x7fd940068710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd960060860 tx=0x7fd950008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.722+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd9480510c0 con 0x7fd960155d80 2023-06-23T18:03:33.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9601f2720 con 0x7fd960155d80 2023-06-23T18:03:33.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd957fff700 1 -- 172.21.15.22:0/211978405 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9480546e0 con 0x7fd960155d80 2023-06-23T18:03:33.901 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:03:33.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 msgr2=0x7fd940068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 0x7fd940068710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd960060860 tx=0x7fd950008030 comp rx=0 tx=0).stop 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 msgr2=0x7fd9601503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.886+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e8a0 tx=0x7fd94800ebb0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 shutdown_connections 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd940066260 0x7fd940068710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd960156b30 0x7fd960156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd960155d80 0x7fd9601503d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 --2- 172.21.15.22:0/211978405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9600cefa0 0x7fd9600b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 >> 172.21.15.22:0/211978405 conn(0x7fd9600b0a20 msgr2=0x7fd960157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 shutdown_connections 2023-06-23T18:03:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:33.890+0000 7fd964952700 1 -- 172.21.15.22:0/211978405 wait complete. 2023-06-23T18:03:33.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:03:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 Processor -- start 2023-06-23T18:03:34.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 -- start start 2023-06-23T18:03:34.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0dbc0cf3a0 0x7f0dbc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 0x7f0dbc0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0ccb30 con 0x7f0dbc0d1f30 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cfcb0 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbbfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0d2880 con 0x7f0dbc0d0730 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0dbc0cf3a0 0x7f0dbc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dba7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0dbc0d1f30 0x7f0dbc0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52856/0 (socket says 172.21.15.22:52856) 2023-06-23T18:03:34.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dba7fc700 1 -- 172.21.15.22:0/206969050 learned_addr learned my addr 172.21.15.22:0/206969050 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbb7fe700 1 --2- 172.21.15.22:0/206969050 >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 0x7f0dbc0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 -- 172.21.15.22:0/206969050 >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 msgr2=0x7f0dbc0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 --2- 172.21.15.22:0/206969050 >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 0x7f0dbc0ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 -- 172.21.15.22:0/206969050 >> v1:172.21.15.22:6789/0 conn(0x7f0dbc0d1f30 legacy=0x7f0dbc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 -- 172.21.15.22:0/206969050 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0d2b60 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbb7fe700 1 --2- 172.21.15.22:0/206969050 >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 0x7f0dbc0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0dbaffd700 1 --2- 172.21.15.22:0/206969050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc0cf770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ceb00 tx=0x7f0dac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde625dfaa146a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0db9ffb700 1 -- 172.21.15.22:0/206969050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac005e50 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0db9ffb700 1 -- 172.21.15.22:0/206969050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac006030 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.010+0000 7f0db9ffb700 1 -- 172.21.15.22:0/206969050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0063c0 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- 172.21.15.22:0/206969050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 msgr2=0x7f0dbc0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/206969050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc0cf770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ceb00 tx=0x7f0dac009b20 comp rx=0 tx=0).stop 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- 172.21.15.22:0/206969050 shutdown_connections 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/206969050 >> v2:172.21.15.169:3300/0 conn(0x7f0dbc0d0730 0x7f0dbc0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/206969050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc0cf770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- 172.21.15.22:0/206969050 >> 172.21.15.22:0/206969050 conn(0x7f0dbc06a280 msgr2=0x7f0dbc09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- 172.21.15.22:0/206969050 shutdown_connections 2023-06-23T18:03:34.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- 172.21.15.22:0/206969050 wait complete. 2023-06-23T18:03:34.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 Processor -- start 2023-06-23T18:03:34.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- start start 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dbc0d0730 0x7f0dbc15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 0x7f0dbc0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0b5a30 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cc2c0 con 0x7f0dbc0d1f30 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbbfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0b5c40 con 0x7f0dbc0d0730 2023-06-23T18:03:34.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49538/0 (socket says 172.21.15.22:49538) 2023-06-23T18:03:34.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.014+0000 7f0dbaffd700 1 -- 172.21.15.22:0/377995801 learned_addr learned my addr 172.21.15.22:0/377995801 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:34.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbb7fe700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 0x7f0dbc0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dba7fc700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dbc0d0730 0x7f0dbc15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 -- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dbc0d0730 msgr2=0x7f0dbc15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dbc0d0730 0x7f0dbc15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 -- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 msgr2=0x7f0dbc0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 0x7f0dbc0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 -- 172.21.15.22:0/377995801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009450 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbb7fe700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 0x7f0dbc0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:34.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbaffd700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0dac013040 tx=0x7f0dac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003980 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac020070 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc0b5f20 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003d20 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc15f040 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7f0dac022030 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.018+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc0b6440 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.022+0000 7f0da37fe700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 0x7f0da8068710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.022+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0dac085130 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.022+0000 7f0dba7fc700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 0x7f0da8068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.022+0000 7f0dba7fc700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 0x7f0da8068710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0b59e0 tx=0x7f0db0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.030+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0dbc0b6440 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0dbc0b6440 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0da37fe700 1 -- 172.21.15.22:0/377995801 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0dbc0b6440 con 0x7f0dbc0cf3a0 2023-06-23T18:03:34.219 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:03:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 msgr2=0x7f0da8068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 0x7f0da8068710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0b59e0 tx=0x7f0db0009040 comp rx=0 tx=0).stop 2023-06-23T18:03:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 msgr2=0x7f0dbc15c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0dac013040 tx=0x7f0dac004770 comp rx=0 tx=0).stop 2023-06-23T18:03:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 shutdown_connections 2023-06-23T18:03:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0da8066260 0x7f0da8068710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dbc0d1f30 0x7f0dbc0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dbc0d0730 0x7f0dbc15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 --2- 172.21.15.22:0/377995801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dbc0cf3a0 0x7f0dbc15c0e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 >> 172.21.15.22:0/377995801 conn(0x7f0dbc06a280 msgr2=0x7f0dbc0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 shutdown_connections 2023-06-23T18:03:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.206+0000 7f0dbbfff700 1 -- 172.21.15.22:0/377995801 wait complete. 2023-06-23T18:03:34.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:03:34.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 Processor -- start 2023-06-23T18:03:34.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 -- start start 2023-06-23T18:03:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe5001603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe500160900 con 0x7fe5000b1400 2023-06-23T18:03:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5000d1730 con 0x7fe5000b3ea0 2023-06-23T18:03:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe506826700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000cce60 con 0x7fe5000cca10 2023-06-23T18:03:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe505824700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe5000b1400 0x7fe5000d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52862/0 (socket says 172.21.15.22:52862) 2023-06-23T18:03:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe505824700 1 -- 172.21.15.22:0/1713307414 learned_addr learned my addr 172.21.15.22:0/1713307414 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe505023700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe5001603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091808994 0 0) 0x7fe5000d1730 con 0x7fe5000b3ea0 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ec0035d0 con 0x7fe5000b3ea0 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911651821 0 0) 0x7fe4f0001110 con 0x7fe5000b1400 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.330+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5000d1730 con 0x7fe5000b1400 2023-06-23T18:03:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428663927 0 0) 0x7fe4ec0035d0 con 0x7fe5000b3ea0 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 msgr2=0x7fe5001603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe5001603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 >> v1:172.21.15.22:6789/0 conn(0x7fe5000b1400 legacy=0x7fe5000d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000cd060 con 0x7fe5000b3ea0 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fe4f80019f0 con 0x7fe5000b3ea0 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 >> v1:172.21.15.169:6789/0 conn(0x7fe5000b3ea0 legacy=0x7fe5000b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4ec003bb0 0x7fe4ec003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe4ec004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec004db0 0x7fe4ec005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000cce60 con 0x7fe4ec003bb0 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec00bae0 con 0x7fe4ec004db0 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec00bc50 con 0x7fe5000cca10 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4ec003bb0 0x7fe4ec003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505023700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe4ec004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506025700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec004db0 0x7fe4ec005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 -- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 msgr2=0x7fe4ec004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5000cca10 0x7fe4ec004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 -- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec004db0 msgr2=0x7fe4ec005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec004db0 0x7fe4ec005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 -- 172.21.15.22:0/1713307414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0035d0 con 0x7fe4ec003bb0 2023-06-23T18:03:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506025700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec004db0 0x7fe4ec005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe505824700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4ec003bb0 0x7fe4ec003fa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0001a30 tx=0x7fe4ec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39519f70562f661d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0004b50 con 0x7fe4ec003bb0 2023-06-23T18:03:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0004cd0 con 0x7fe4ec003bb0 2023-06-23T18:03:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe504822700 1 -- 172.21.15.22:0/1713307414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0007310 con 0x7fe4ec003bb0 2023-06-23T18:03:34.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 -- 172.21.15.22:0/1713307414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4ec003bb0 msgr2=0x7fe4ec003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 --2- 172.21.15.22:0/1713307414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4ec003bb0 0x7fe4ec003fa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0001a30 tx=0x7fe4ec0058a0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506025700 1 -- 172.21.15.22:0/1713307414 reap_dead start 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 -- 172.21.15.22:0/1713307414 shutdown_connections 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 -- 172.21.15.22:0/1713307414 >> 172.21.15.22:0/1713307414 conn(0x7fe500069960 msgr2=0x7fe500069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 -- 172.21.15.22:0/1713307414 shutdown_connections 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.334+0000 7fe506826700 1 -- 172.21.15.22:0/1713307414 wait complete. 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 Processor -- start 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 -- start start 2023-06-23T18:03:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ec003bb0 0x7fe50014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 0x7fe5000b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe500157210 con 0x7fe500156860 2023-06-23T18:03:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b6d70 con 0x7fe5000b5cf0 2023-06-23T18:03:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506826700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b67b0 con 0x7fe4ec003bb0 2023-06-23T18:03:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe505824700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ec003bb0 0x7fe50014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49558/0 (socket says 172.21.15.22:49558) 2023-06-23T18:03:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 -- 172.21.15.22:0/2146759016 learned_addr learned my addr 172.21.15.22:0/2146759016 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe505023700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 0x7fe5000b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 -- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ec003bb0 msgr2=0x7fe50014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ec003bb0 0x7fe50014e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 -- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 msgr2=0x7fe5000b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 0x7fe5000b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 -- 172.21.15.22:0/2146759016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000b6a90 con 0x7fe500156860 2023-06-23T18:03:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe505023700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 0x7fe5000b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe506025700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8001330 tx=0x7fe4f800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8003e30 con 0x7fe500156860 2023-06-23T18:03:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.338+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f80020e0 con 0x7fe500156860 2023-06-23T18:03:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5000ca0b0 con 0x7fe500156860 2023-06-23T18:03:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800bb20 con 0x7fe500156860 2023-06-23T18:03:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5000b76d0 con 0x7fe500156860 2023-06-23T18:03:34.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5000c9f10 con 0x7fe500156860 2023-06-23T18:03:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 85602+0+0 (secure 0 0 0) 0x7fe4f800bc80 con 0x7fe500156860 2023-06-23T18:03:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe4f67fc700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 0x7fe4e4068710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.342+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe4f8083300 con 0x7fe500156860 2023-06-23T18:03:34.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.354+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe5000c9f10 con 0x7fe500156860 2023-06-23T18:03:34.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.362+0000 7fe505824700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 0x7fe4e4068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.362+0000 7fe505824700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 0x7fe4e4068710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.374+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe4f8050f00 con 0x7fe500156860 2023-06-23T18:03:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.558+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5000c9f10 con 0x7fe500156860 2023-06-23T18:03:34.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.630+0000 7fe4f67fc700 1 -- 172.21.15.22:0/2146759016 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fe4f8018de0 con 0x7fe500156860 2023-06-23T18:03:34.646 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:34.647 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T18:03:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 msgr2=0x7fe4e4068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 0x7fe4e4068710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0007000 comp rx=0 tx=0).stop 2023-06-23T18:03:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 msgr2=0x7fe500156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8001330 tx=0x7fe4f800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 shutdown_connections 2023-06-23T18:03:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe500156860 0x7fe500156cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe4e4066260 0x7fe4e4068710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5000b5cf0 0x7fe5000b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 --2- 172.21.15.22:0/2146759016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ec003bb0 0x7fe50014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 >> 172.21.15.22:0/2146759016 conn(0x7fe500069960 msgr2=0x7fe5000723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 shutdown_connections 2023-06-23T18:03:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.638+0000 7fe506826700 1 -- 172.21.15.22:0/2146759016 wait complete. 2023-06-23T18:03:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2023-06-23T18:03:34.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:34.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.766+0000 7f610632d700 1 Processor -- start 2023-06-23T18:03:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.766+0000 7f610632d700 1 -- start start 2023-06-23T18:03:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61000b7ec0 con 0x7f61000cc9e0 2023-06-23T18:03:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61000b1ad0 con 0x7f61000b7af0 2023-06-23T18:03:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61000d0a10 con 0x7f61000cf480 2023-06-23T18:03:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49576/0 (socket says 172.21.15.22:49576) 2023-06-23T18:03:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6105b2c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61000cc9e0 0x7f61000d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52868/0 (socket says 172.21.15.22:52868) 2023-06-23T18:03:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 -- 172.21.15.22:0/363129803 learned_addr learned my addr 172.21.15.22:0/363129803 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610532b700 1 --1- 172.21.15.22:0/363129803 >> v1:172.21.15.169:6789/0 conn(0x7f61000b7af0 0x7f61000ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:34014/0 (socket says 172.21.15.22:34014) 2023-06-23T18:03:34.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 -- 172.21.15.22:0/363129803 >> v1:172.21.15.169:6789/0 conn(0x7f61000b7af0 legacy=0x7f61000ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 -- 172.21.15.22:0/363129803 >> v1:172.21.15.22:6789/0 conn(0x7f61000cc9e0 legacy=0x7f61000d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 -- 172.21.15.22:0/363129803 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000d0cb0 con 0x7f61000cf480 2023-06-23T18:03:34.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f6104b2a700 1 --2- 172.21.15.22:0/363129803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009910 tx=0x7f60f0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f974110a804f462 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f60f7fff700 1 -- 172.21.15.22:0/363129803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f000ba60 con 0x7f61000cf480 2023-06-23T18:03:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f60f7fff700 1 -- 172.21.15.22:0/363129803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f000bbc0 con 0x7f61000cf480 2023-06-23T18:03:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f60f7fff700 1 -- 172.21.15.22:0/363129803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009e60 con 0x7f61000cf480 2023-06-23T18:03:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 -- 172.21.15.22:0/363129803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 msgr2=0x7f61000ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.770+0000 7f610632d700 1 --2- 172.21.15.22:0/363129803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009910 tx=0x7f60f0009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 -- 172.21.15.22:0/363129803 shutdown_connections 2023-06-23T18:03:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 --2- 172.21.15.22:0/363129803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 -- 172.21.15.22:0/363129803 >> 172.21.15.22:0/363129803 conn(0x7f6100069960 msgr2=0x7f6100069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 -- 172.21.15.22:0/363129803 shutdown_connections 2023-06-23T18:03:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 -- 172.21.15.22:0/363129803 wait complete. 2023-06-23T18:03:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 Processor -- start 2023-06-23T18:03:34.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.774+0000 7f610632d700 1 -- start start 2023-06-23T18:03:34.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 0x7f61001497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61000cc9e0 0x7f610015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610015c380 con 0x7f61000cf480 2023-06-23T18:03:34.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610015c660 con 0x7f61000cc9e0 2023-06-23T18:03:34.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610015c940 con 0x7f61000b7af0 2023-06-23T18:03:34.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49590/0 (socket says 172.21.15.22:49590) 2023-06-23T18:03:34.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 -- 172.21.15.22:0/3136086574 learned_addr learned my addr 172.21.15.22:0/3136086574 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:34.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6104b2a700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61000cc9e0 0x7f610015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610532b700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 0x7f61001497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 -- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 msgr2=0x7f61001497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 0x7f61001497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 -- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61000cc9e0 msgr2=0x7f610015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61000cc9e0 0x7f610015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 -- 172.21.15.22:0/3136086574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009570 con 0x7f61000cf480 2023-06-23T18:03:34.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610532b700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 0x7f61001497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:34.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f6105b2c700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c500 tx=0x7f60f800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8005cd0 con 0x7f61000cf480 2023-06-23T18:03:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f8005e30 con 0x7f61000cf480 2023-06-23T18:03:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f80072f0 con 0x7f61000cf480 2023-06-23T18:03:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610015cc20 con 0x7f61000cf480 2023-06-23T18:03:34.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610015d680 con 0x7f61000cf480 2023-06-23T18:03:34.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610015ce60 con 0x7f61000cf480 2023-06-23T18:03:34.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.778+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f60f8007450 con 0x7f61000cf480 2023-06-23T18:03:34.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.782+0000 7f60f5ffb700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 0x7f60e4068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:34.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.782+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f60f8082ea0 con 0x7f61000cf480 2023-06-23T18:03:34.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.782+0000 7f610532b700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 0x7f60e4068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.782+0000 7f610532b700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 0x7f60e4068820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0012e0 tx=0x7f60fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.786+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f610015ce60 con 0x7f61000cf480 2023-06-23T18:03:34.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.958+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f610015ce60 con 0x7f61000cf480 2023-06-23T18:03:34.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.958+0000 7f60f5ffb700 1 -- 172.21.15.22:0/3136086574 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14624 (secure 0 0 0) 0x7f610015ce60 con 0x7f61000cf480 2023-06-23T18:03:34.974 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:34.974 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:34.235706+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:34.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 msgr2=0x7f60e4068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 0x7f60e4068820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0012e0 tx=0x7f60fc009040 comp rx=0 tx=0).stop 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 msgr2=0x7f61000ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c500 tx=0x7f60f800c810 comp rx=0 tx=0).stop 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 shutdown_connections 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61000cf480 0x7f61000ce540 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f60e4066370 0x7f60e4068820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61000cc9e0 0x7f610015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 --2- 172.21.15.22:0/3136086574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61000b7af0 0x7f61001497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:34.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 >> 172.21.15.22:0/3136086574 conn(0x7f6100069960 msgr2=0x7f6100069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:34.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 shutdown_connections 2023-06-23T18:03:34.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:34.962+0000 7f610632d700 1 -- 172.21.15.22:0/3136086574 wait complete. 2023-06-23T18:03:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:03:34.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:03:35.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 Processor -- start 2023-06-23T18:03:35.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 -- start start 2023-06-23T18:03:35.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcec0cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdcec0cca20 0x7fdcec0d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec0d1d40 con 0x7fdcec0d0a20 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec0b1cf0 con 0x7fdcec0cca20 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1d13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0d31e0 con 0x7fdcec0b1920 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1512700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdcec0d0a20 0x7fdcec15f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52876/0 (socket says 172.21.15.22:52876) 2023-06-23T18:03:35.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.098+0000 7fdcf1512700 1 -- 172.21.15.22:0/3223026877 learned_addr learned my addr 172.21.15.22:0/3223026877 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:35.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcebfff700 1 --2- 172.21.15.22:0/3223026877 >> v2:172.21.15.169:3300/0 conn(0x7fdcec0cca20 0x7fdcec0d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcec0cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445192113 0 0) 0x7fdcec0d1d40 con 0x7fdcec0d0a20 2023-06-23T18:03:35.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd40035d0 con 0x7fdcec0d0a20 2023-06-23T18:03:35.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013973577 0 0) 0x7fdcd40035d0 con 0x7fdcec0d0a20 2023-06-23T18:03:35.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 msgr2=0x7fdcec0cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcec0cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 >> v2:172.21.15.169:3300/0 conn(0x7fdcec0cca20 msgr2=0x7fdcec0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 --2- 172.21.15.22:0/3223026877 >> v2:172.21.15.169:3300/0 conn(0x7fdcec0cca20 0x7fdcec0d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0cf130 con 0x7fdcec0d0a20 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fdce40019f0 con 0x7fdcec0d0a20 2023-06-23T18:03:35.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 >> v1:172.21.15.22:6789/0 conn(0x7fdcec0d0a20 legacy=0x7fdcec15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcd4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcd4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcd4004db0 0x7fdcd4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0b1cf0 con 0x7fdcd4003a20 2023-06-23T18:03:35.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0d31e0 con 0x7fdcd4004db0 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd400ba30 con 0x7fdcec0b1920 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcd4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcebfff700 1 --2- 172.21.15.22:0/3223026877 >> v2:172.21.15.169:3300/0 conn(0x7fdcec0cca20 0x7fdcec0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1512700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcd4004db0 0x7fdcd4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcebfff700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcd4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 -- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 msgr2=0x7fdcd4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0b1920 0x7fdcd4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 -- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcd4004db0 msgr2=0x7fdcd4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcd4004db0 0x7fdcd4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 -- 172.21.15.22:0/3223026877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40035d0 con 0x7fdcd4003a20 2023-06-23T18:03:35.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcd4003e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009b70 tx=0x7fdcd40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3452450b3c29e9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000e660 con 0x7fdcd4003a20 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdceb7fe700 1 -- 172.21.15.22:0/3223026877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce000e7c0 con 0x7fdcd4003a20 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 -- 172.21.15.22:0/3223026877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 msgr2=0x7fdcd4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/3223026877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcd4003e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009b70 tx=0x7fdcd40058a0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1512700 1 -- 172.21.15.22:0/3223026877 reap_dead start 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 -- 172.21.15.22:0/3223026877 shutdown_connections 2023-06-23T18:03:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 -- 172.21.15.22:0/3223026877 >> 172.21.15.22:0/3223026877 conn(0x7fdcec069960 msgr2=0x7fdcec069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 -- 172.21.15.22:0/3223026877 shutdown_connections 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.102+0000 7fdcf1d13700 1 -- 172.21.15.22:0/3223026877 wait complete. 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 Processor -- start 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- start start 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 0x7fdcec0c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 0x7fdcec0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0cbbb0 con 0x7fdcd4003a20 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0cc440 con 0x7fdcec14c880 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec157280 con 0x7fdcec0cbfd0 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49604/0 (socket says 172.21.15.22:49604) 2023-06-23T18:03:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 -- 172.21.15.22:0/1753238810 learned_addr learned my addr 172.21.15.22:0/1753238810 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:35.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcebfff700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 0x7fdcec0c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1512700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 0x7fdcec0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 -- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 msgr2=0x7fdcec0c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 0x7fdcec0c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 -- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 msgr2=0x7fdcec0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 0x7fdcec0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 -- 172.21.15.22:0/1753238810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0caf30 con 0x7fdcd4003a20 2023-06-23T18:03:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1512700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 0x7fdcec0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcebfff700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 0x7fdcec0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf0d11700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdce00090b0 tx=0x7fdce00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000e660 con 0x7fdcd4003a20 2023-06-23T18:03:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce0013e10 con 0x7fdcd4003a20 2023-06-23T18:03:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec0cb210 con 0x7fdcd4003a20 2023-06-23T18:03:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0004270 con 0x7fdcd4003a20 2023-06-23T18:03:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec0cb7d0 con 0x7fdcd4003a20 2023-06-23T18:03:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.106+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec04e4c0 con 0x7fdcd4003a20 2023-06-23T18:03:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.110+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdce00043d0 con 0x7fdcd4003a20 2023-06-23T18:03:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.110+0000 7fdce97fa700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 0x7fdcd80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.110+0000 7fdcebfff700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 0x7fdcd80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.110+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdce00853e0 con 0x7fdcd4003a20 2023-06-23T18:03:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.110+0000 7fdcebfff700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 0x7fdcd80687a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0c8930 tx=0x7fdcdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.118+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fdce0052cc0 con 0x7fdcd4003a20 2023-06-23T18:03:35.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.278+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcec0b46d0 con 0x7fdcd4003a20 2023-06-23T18:03:35.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.278+0000 7fdce97fa700 1 -- 172.21.15.22:0/1753238810 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdce00562e0 con 0x7fdcd4003a20 2023-06-23T18:03:35.293 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 msgr2=0x7fdcd80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 0x7fdcd80687a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0c8930 tx=0x7fdcdc008040 comp rx=0 tx=0).stop 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 msgr2=0x7fdcec0ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdce00090b0 tx=0x7fdce00047e0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 shutdown_connections 2023-06-23T18:03:35.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdcd80662f0 0x7fdcd80687a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcec14c880 0x7fdcec0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcec0cbfd0 0x7fdcec0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 --2- 172.21.15.22:0/1753238810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcd4003a20 0x7fdcec0ca950 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdce00090b0 tx=0x7fdce00047e0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 >> 172.21.15.22:0/1753238810 conn(0x7fdcec069960 msgr2=0x7fdcec160000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 shutdown_connections 2023-06-23T18:03:35.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.282+0000 7fdcf1d13700 1 -- 172.21.15.22:0/1753238810 wait complete. 2023-06-23T18:03:35.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:03:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.410+0000 7ffa1359e700 1 Processor -- start 2023-06-23T18:03:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.410+0000 7ffa1359e700 1 -- start start 2023-06-23T18:03:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.410+0000 7ffa1359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa140b1920 0x7ffa140cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.410+0000 7ffa1359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa140d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.410+0000 7ffa1359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa140d0a20 0x7ffa1415f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa140b1cf0 con 0x7ffa140b1920 2023-06-23T18:03:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa140d1d40 con 0x7ffa140d0a20 2023-06-23T18:03:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140cf130 con 0x7ffa140cca20 2023-06-23T18:03:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa140b1920 0x7ffa140cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa140b1920 0x7ffa140cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49606/0 (socket says 172.21.15.22:49606) 2023-06-23T18:03:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 -- 172.21.15.22:0/2218529504 learned_addr learned my addr 172.21.15.22:0/2218529504 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa11d9b700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa140d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa12d9d700 1 --2- 172.21.15.22:0/2218529504 >> v2:172.21.15.169:3300/0 conn(0x7ffa140d0a20 0x7ffa1415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 -- 172.21.15.22:0/2218529504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 msgr2=0x7ffa140d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa140d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 -- 172.21.15.22:0/2218529504 >> v2:172.21.15.169:3300/0 conn(0x7ffa140d0a20 msgr2=0x7ffa1415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2218529504 >> v2:172.21.15.169:3300/0 conn(0x7ffa140d0a20 0x7ffa1415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 -- 172.21.15.22:0/2218529504 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140cf410 con 0x7ffa140b1920 2023-06-23T18:03:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140b1920 0x7ffa140cffa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa14160780 tx=0x7ffa0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b88cc4ef638ee7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1159a700 1 -- 172.21.15.22:0/2218529504 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400e070 con 0x7ffa140b1920 2023-06-23T18:03:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1159a700 1 -- 172.21.15.22:0/2218529504 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0400b910 con 0x7ffa140b1920 2023-06-23T18:03:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1159a700 1 -- 172.21.15.22:0/2218529504 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400bcc0 con 0x7ffa140b1920 2023-06-23T18:03:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- 172.21.15.22:0/2218529504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140b1920 msgr2=0x7ffa140cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140b1920 0x7ffa140cffa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa14160780 tx=0x7ffa0400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- 172.21.15.22:0/2218529504 shutdown_connections 2023-06-23T18:03:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2218529504 >> v2:172.21.15.169:3300/0 conn(0x7ffa140d0a20 0x7ffa1415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa140d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2218529504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140b1920 0x7ffa140cffa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- 172.21.15.22:0/2218529504 >> 172.21.15.22:0/2218529504 conn(0x7ffa14069960 msgr2=0x7ffa14069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- 172.21.15.22:0/2218529504 shutdown_connections 2023-06-23T18:03:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.414+0000 7ffa1359e700 1 -- 172.21.15.22:0/2218529504 wait complete. 2023-06-23T18:03:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 Processor -- start 2023-06-23T18:03:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- start start 2023-06-23T18:03:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa141f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa140b5f60 0x7ffa140c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140b8200 con 0x7ffa140d0a20 2023-06-23T18:03:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141f1280 con 0x7ffa140b5f60 2023-06-23T18:03:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140b7d70 con 0x7ffa140cca20 2023-06-23T18:03:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49616/0 (socket says 172.21.15.22:49616) 2023-06-23T18:03:35.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 -- 172.21.15.22:0/2866949575 learned_addr learned my addr 172.21.15.22:0/2866949575 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:35.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa12d9d700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa140b5f60 0x7ffa140c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa141f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 -- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 msgr2=0x7ffa141f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa141f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 -- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa140b5f60 msgr2=0x7ffa140c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa140b5f60 0x7ffa140c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 -- 172.21.15.22:0/2866949575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009b30 con 0x7ffa140d0a20 2023-06-23T18:03:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa11d9b700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00ec10 tx=0x7ff9fc009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc009740 con 0x7ffa140d0a20 2023-06-23T18:03:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc00c880 con 0x7ffa140d0a20 2023-06-23T18:03:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa140b7160 con 0x7ffa140d0a20 2023-06-23T18:03:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00cc10 con 0x7ffa140d0a20 2023-06-23T18:03:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.418+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa140b7720 con 0x7ffa140d0a20 2023-06-23T18:03:35.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.422+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa140b7f20 con 0x7ffa140d0a20 2023-06-23T18:03:35.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.422+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9fc00cd70 con 0x7ffa140d0a20 2023-06-23T18:03:35.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.422+0000 7ffa0affd700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 0x7ff9f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.422+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff9fc083aa0 con 0x7ffa140d0a20 2023-06-23T18:03:35.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.422+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 0x7ff9f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.426+0000 7ffa1259c700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 0x7ff9f80687a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa1408b3d0 tx=0x7ffa04009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.430+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ffa140b7f20 con 0x7ffa140d0a20 2023-06-23T18:03:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa140b7ee0 con 0x7ffa140d0a20 2023-06-23T18:03:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa0affd700 1 -- 172.21.15.22:0/2866949575 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9fc0170a0 con 0x7ffa140d0a20 2023-06-23T18:03:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 msgr2=0x7ff9f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 0x7ff9f80687a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa1408b3d0 tx=0x7ffa04009000 comp rx=0 tx=0).stop 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 msgr2=0x7ffa140b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00ec10 tx=0x7ff9fc009e00 comp rx=0 tx=0).stop 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 shutdown_connections 2023-06-23T18:03:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f80662f0 0x7ff9f80687a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa140b5f60 0x7ffa140c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa140d0a20 0x7ffa140b68c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 --2- 172.21.15.22:0/2866949575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa140cca20 0x7ffa141f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 >> 172.21.15.22:0/2866949575 conn(0x7ffa14069960 msgr2=0x7ffa140b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 shutdown_connections 2023-06-23T18:03:35.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.594+0000 7ffa1359e700 1 -- 172.21.15.22:0/2866949575 wait complete. 2023-06-23T18:03:35.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T18:03:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.726+0000 7f8e7759e700 1 Processor -- start 2023-06-23T18:03:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.726+0000 7f8e7759e700 1 -- start start 2023-06-23T18:03:35.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e780cf480 0x7f8e780ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e780b7ec0 con 0x7f8e780b7af0 2023-06-23T18:03:35.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e780b1ad0 con 0x7f8e780cc9e0 2023-06-23T18:03:35.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e780d0a10 con 0x7f8e780cf480 2023-06-23T18:03:35.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e75d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e780cf480 0x7f8e780ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7659c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8e780b7af0 0x7f8e780ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52878/0 (socket says 172.21.15.22:52878) 2023-06-23T18:03:35.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7659c700 1 -- 172.21.15.22:0/1392921692 learned_addr learned my addr 172.21.15.22:0/1392921692 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e75d9b700 1 -- 172.21.15.22:0/1392921692 >> v1:172.21.15.169:6789/0 conn(0x7f8e780cc9e0 legacy=0x7f8e780d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e75d9b700 1 -- 172.21.15.22:0/1392921692 >> v1:172.21.15.22:6789/0 conn(0x7f8e780b7af0 legacy=0x7f8e780ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7559a700 1 -- 172.21.15.22:0/1392921692 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731518366 0 0) 0x7f8e780b7ec0 con 0x7f8e780b7af0 2023-06-23T18:03:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e75d9b700 1 -- 172.21.15.22:0/1392921692 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e780d0cb0 con 0x7f8e780cf480 2023-06-23T18:03:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e75d9b700 1 --2- 172.21.15.22:0/1392921692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780cf480 0x7f8e780ce540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009850 tx=0x7f8e6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9fa71604a843f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7559a700 1 -- 172.21.15.22:0/1392921692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c011830 con 0x7f8e780cf480 2023-06-23T18:03:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7559a700 1 -- 172.21.15.22:0/1392921692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c011990 con 0x7f8e780cf480 2023-06-23T18:03:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7559a700 1 -- 172.21.15.22:0/1392921692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c011d20 con 0x7f8e780cf480 2023-06-23T18:03:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- 172.21.15.22:0/1392921692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780cf480 msgr2=0x7f8e780ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 --2- 172.21.15.22:0/1392921692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780cf480 0x7f8e780ce540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009850 tx=0x7f8e6c009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:35.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- 172.21.15.22:0/1392921692 shutdown_connections 2023-06-23T18:03:35.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 --2- 172.21.15.22:0/1392921692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780cf480 0x7f8e780ce540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- 172.21.15.22:0/1392921692 >> 172.21.15.22:0/1392921692 conn(0x7f8e78069960 msgr2=0x7f8e78069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- 172.21.15.22:0/1392921692 shutdown_connections 2023-06-23T18:03:35.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.730+0000 7f8e7759e700 1 -- 172.21.15.22:0/1392921692 wait complete. 2023-06-23T18:03:35.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 Processor -- start 2023-06-23T18:03:35.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- start start 2023-06-23T18:03:35.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e780cc9e0 0x7f8e781f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780cf480 0x7f8e780b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781f2460 con 0x7f8e780b7af0 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781f1ea0 con 0x7f8e780cc9e0 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781f2180 con 0x7f8e780cf480 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49634/0 (socket says 172.21.15.22:49634) 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 -- 172.21.15.22:0/2833783192 learned_addr learned my addr 172.21.15.22:0/2833783192 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e75d9b700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e780cc9e0 0x7f8e781f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e76d9d700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780cf480 0x7f8e780b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 -- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780cf480 msgr2=0x7f8e780b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780cf480 0x7f8e780b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 -- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e780cc9e0 msgr2=0x7f8e781f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e780cc9e0 0x7f8e781f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 -- 172.21.15.22:0/2833783192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c0094b0 con 0x7f8e780b7af0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7659c700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800c550 tx=0x7f8e6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68004ca0 con 0x7f8e780b7af0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68004e00 con 0x7f8e780b7af0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e781f27b0 con 0x7f8e780b7af0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007310 con 0x7f8e780b7af0 2023-06-23T18:03:35.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.734+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7814c5e0 con 0x7f8e780b7af0 2023-06-23T18:03:35.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.738+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e781f2610 con 0x7f8e780b7af0 2023-06-23T18:03:35.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.738+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8e68007470 con 0x7f8e780b7af0 2023-06-23T18:03:35.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.738+0000 7f8e66ffd700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 0x7f8e580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:35.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.738+0000 7f8e75d9b700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 0x7f8e580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:35.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.738+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e68083520 con 0x7f8e780b7af0 2023-06-23T18:03:35.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.742+0000 7f8e75d9b700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 0x7f8e580687a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c014040 tx=0x7f8e6c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:35.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.746+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8e781f2610 con 0x7f8e780b7af0 2023-06-23T18:03:35.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.910+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8e780d1250 con 0x7f8e780b7af0 2023-06-23T18:03:35.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.910+0000 7f8e66ffd700 1 -- 172.21.15.22:0/2833783192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8e780d1250 con 0x7f8e780b7af0 2023-06-23T18:03:35.927 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:35.927 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2023-06-23T18:03:35.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:03:35.927 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2023-06-23T18:03:35.927 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2023-06-23T18:03:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:03:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:03:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 2, 2023-06-23T18:03:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": "rbd" 2023-06-23T18:03:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:03:35.929 INFO:teuthology.orchestra.run.smithi022.stdout:] 2023-06-23T18:03:35.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 msgr2=0x7f8e580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 0x7f8e580687a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c014040 tx=0x7f8e6c008030 comp rx=0 tx=0).stop 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 msgr2=0x7f8e7814c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800c550 tx=0x7f8e6800c860 comp rx=0 tx=0).stop 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 shutdown_connections 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e780cf480 0x7f8e780b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8e580662f0 0x7f8e580687a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e780cc9e0 0x7f8e781f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 --2- 172.21.15.22:0/2833783192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e780b7af0 0x7f8e7814c0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:35.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 >> 172.21.15.22:0/2833783192 conn(0x7f8e78069960 msgr2=0x7f8e7815b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:35.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 shutdown_connections 2023-06-23T18:03:35.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:35.914+0000 7f8e7759e700 1 -- 172.21.15.22:0/2833783192 wait complete. 2023-06-23T18:03:35.940 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T18:03:35.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T18:03:36.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 Processor -- start 2023-06-23T18:03:36.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 -- start start 2023-06-23T18:03:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba880b45b0 0x7fba881603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba88160900 con 0x7fba880b1400 2023-06-23T18:03:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba880d1730 con 0x7fba880cca10 2023-06-23T18:03:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba87fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba880cd060 con 0x7fba880b45b0 2023-06-23T18:03:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba86ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fba880b1400 0x7fba880d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52886/0 (socket says 172.21.15.22:52886) 2023-06-23T18:03:36.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba86ffd700 1 -- 172.21.15.22:0/464088684 learned_addr learned my addr 172.21.15.22:0/464088684 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:36.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.046+0000 7fba877fe700 1 --2- 172.21.15.22:0/464088684 >> v2:172.21.15.169:3300/0 conn(0x7fba880b45b0 0x7fba881603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944206214 0 0) 0x7fba78001110 con 0x7fba880b1400 2023-06-23T18:03:36.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba740035d0 con 0x7fba880b1400 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332821814 0 0) 0x7fba880d1730 con 0x7fba880cca10 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba78001110 con 0x7fba880cca10 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba877fe700 1 -- 172.21.15.22:0/464088684 >> v1:172.21.15.169:6789/0 conn(0x7fba880cca10 legacy=0x7fba880b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba877fe700 1 -- 172.21.15.22:0/464088684 >> v1:172.21.15.22:6789/0 conn(0x7fba880b1400 legacy=0x7fba880d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba877fe700 1 -- 172.21.15.22:0/464088684 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880cd340 con 0x7fba880b45b0 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba877fe700 1 --2- 172.21.15.22:0/464088684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b45b0 0x7fba881603c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba80009d90 tx=0x7fba8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc1eeb5f738e208 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8000e070 con 0x7fba880b45b0 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8000bb20 con 0x7fba880b45b0 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba85ffb700 1 -- 172.21.15.22:0/464088684 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80012430 con 0x7fba880b45b0 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 -- 172.21.15.22:0/464088684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b45b0 msgr2=0x7fba881603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 --2- 172.21.15.22:0/464088684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b45b0 0x7fba881603c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba80009d90 tx=0x7fba8000b330 comp rx=0 tx=0).stop 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 -- 172.21.15.22:0/464088684 shutdown_connections 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 --2- 172.21.15.22:0/464088684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b45b0 0x7fba881603c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 -- 172.21.15.22:0/464088684 >> 172.21.15.22:0/464088684 conn(0x7fba88069960 msgr2=0x7fba88069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:36.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 -- 172.21.15.22:0/464088684 shutdown_connections 2023-06-23T18:03:36.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.050+0000 7fba87fff700 1 -- 172.21.15.22:0/464088684 wait complete. 2023-06-23T18:03:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 Processor -- start 2023-06-23T18:03:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- start start 2023-06-23T18:03:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b1400 0x7fba881f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 0x7fba88159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba881f5c40 con 0x7fba880b45b0 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880cbdf0 con 0x7fba880b1400 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880ca810 con 0x7fba880cca10 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49646/0 (socket says 172.21.15.22:49646) 2023-06-23T18:03:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 -- 172.21.15.22:0/2291917018 learned_addr learned my addr 172.21.15.22:0/2291917018 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba86ffd700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b1400 0x7fba881f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba877fe700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 0x7fba88159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 -- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 msgr2=0x7fba88159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 0x7fba88159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 -- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b1400 msgr2=0x7fba881f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b1400 0x7fba881f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 -- 172.21.15.22:0/2291917018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009a70 con 0x7fba880b45b0 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba877fe700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 0x7fba88159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba867fc700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c490 tx=0x7fba7000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba700052d0 con 0x7fba880b45b0 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70005450 con 0x7fba880b45b0 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba880caaf0 con 0x7fba880b45b0 2023-06-23T18:03:36.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba700038e0 con 0x7fba880b45b0 2023-06-23T18:03:36.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.054+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba880cb0b0 con 0x7fba880b45b0 2023-06-23T18:03:36.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba880b7990 con 0x7fba880b45b0 2023-06-23T18:03:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fba70003a40 con 0x7fba880b45b0 2023-06-23T18:03:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba7ffff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 0x7fba680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba86ffd700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 0x7fba680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba70082e00 con 0x7fba880b45b0 2023-06-23T18:03:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.058+0000 7fba86ffd700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 0x7fba680687a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba78007fd0 tx=0x7fba78007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.066+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fba700505c0 con 0x7fba880b45b0 2023-06-23T18:03:36.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.226+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fba8804e4c0 con 0x7fba880b45b0 2023-06-23T18:03:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.338+0000 7fba7ffff700 1 -- 172.21.15.22:0/2291917018 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fba70053be0 con 0x7fba880b45b0 2023-06-23T18:03:36.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.346+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 msgr2=0x7fba680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.346+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 0x7fba680687a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba78007fd0 tx=0x7fba78007de0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.346+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 msgr2=0x7fba880cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.346+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c490 tx=0x7fba7000c7a0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 shutdown_connections 2023-06-23T18:03:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba880cca10 0x7fba88159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba880b45b0 0x7fba880cb8b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba680662f0 0x7fba680687a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 --2- 172.21.15.22:0/2291917018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba880b1400 0x7fba881f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 >> 172.21.15.22:0/2291917018 conn(0x7fba88069960 msgr2=0x7fba8815f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 shutdown_connections 2023-06-23T18:03:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.350+0000 7fba87fff700 1 -- 172.21.15.22:0/2291917018 wait complete. 2023-06-23T18:03:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2023-06-23T18:03:36.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-06-23T18:03:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.478+0000 7f5db1402700 1 Processor -- start 2023-06-23T18:03:36.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.478+0000 7f5db1402700 1 -- start start 2023-06-23T18:03:36.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac0b7ba0 0x7f5dac0b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5dac0d13f0 0x7f5dac0d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0d1dd0 con 0x7f5dac15c4b0 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac0cd760 con 0x7f5dac0d13f0 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0b5920 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5dac0d13f0 0x7f5dac0d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5dac0d13f0 0x7f5dac0d1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40212/0 (socket says 172.21.15.22:40212) 2023-06-23T18:03:36.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 -- 172.21.15.22:0/1643865777 learned_addr learned my addr 172.21.15.22:0/1643865777 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5da37fe700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac0b7ba0 0x7f5dac0b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 -- 172.21.15.22:0/1643865777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac0b7ba0 msgr2=0x7f5dac0b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac0b7ba0 0x7f5dac0b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 -- 172.21.15.22:0/1643865777 >> v1:172.21.15.169:6789/0 conn(0x7f5dac15c4b0 legacy=0x7f5dac0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 -- 172.21.15.22:0/1643865777 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0d0350 con 0x7f5dac0d13f0 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db0c01700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0d13f0 0x7f5dac0d1890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e80 tx=0x7f5da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891621fdf3d07b1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5dab7fe700 1 -- 172.21.15.22:0/1643865777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400e070 con 0x7f5dac0d13f0 2023-06-23T18:03:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5dab7fe700 1 -- 172.21.15.22:0/1643865777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da400b8d0 con 0x7f5dac0d13f0 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5dab7fe700 1 -- 172.21.15.22:0/1643865777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400bc80 con 0x7f5dac0d13f0 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- 172.21.15.22:0/1643865777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0d13f0 msgr2=0x7f5dac0d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0d13f0 0x7f5dac0d1890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e80 tx=0x7f5da400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- 172.21.15.22:0/1643865777 shutdown_connections 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0d13f0 0x7f5dac0d1890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 --2- 172.21.15.22:0/1643865777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac0b7ba0 0x7f5dac0b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- 172.21.15.22:0/1643865777 >> 172.21.15.22:0/1643865777 conn(0x7f5dac069960 msgr2=0x7f5dac069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:36.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- 172.21.15.22:0/1643865777 shutdown_connections 2023-06-23T18:03:36.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.482+0000 7f5db1402700 1 -- 172.21.15.22:0/1643865777 wait complete. 2023-06-23T18:03:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 Processor -- start 2023-06-23T18:03:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 -- start start 2023-06-23T18:03:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac0d13f0 0x7f5dac152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac15c4b0 0x7f5dac0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c9830 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c9b10 con 0x7f5dac0d13f0 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db1402700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c9df0 con 0x7f5dac15c4b0 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40228/0 (socket says 172.21.15.22:40228) 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5da37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac0d13f0 0x7f5dac152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5dabfff700 1 -- 172.21.15.22:0/1163596223 learned_addr learned my addr 172.21.15.22:0/1163596223 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.486+0000 7f5db0c01700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac15c4b0 0x7f5dac0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 -- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac15c4b0 msgr2=0x7f5dac0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac15c4b0 0x7f5dac0ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 -- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac0d13f0 msgr2=0x7f5dac152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac0d13f0 0x7f5dac152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 -- 172.21.15.22:0/1163596223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4009b30 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5dabfff700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c500 tx=0x7f5d9c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0052d0 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c005430 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac0ca0d0 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0055a0 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac153120 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dac04e4c0 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5d9c0036a0 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.490+0000 7f5da97fa700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 0x7f5d900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:36.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.494+0000 7f5da37fe700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 0x7f5d900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:36.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.494+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f5d9c082d00 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.494+0000 7f5da37fe700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 0x7f5d900687a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0cc490 tx=0x7f5d94009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.502+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5d9c050360 con 0x7f5dac0b7ba0 2023-06-23T18:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:36.662+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f5dac0d0ec0 con 0x7f5dac0b7ba0 2023-06-23T18:03:37.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.350+0000 7f5da97fa700 1 -- 172.21.15.22:0/1163596223 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f5d9c053980 con 0x7f5dac0b7ba0 2023-06-23T18:03:37.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.358+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 msgr2=0x7f5d900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.358+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 0x7f5d900687a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0cc490 tx=0x7f5d94009240 comp rx=0 tx=0).stop 2023-06-23T18:03:37.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.358+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 msgr2=0x7f5dac15eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.358+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c500 tx=0x7f5d9c00c810 comp rx=0 tx=0).stop 2023-06-23T18:03:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 shutdown_connections 2023-06-23T18:03:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dac15c4b0 0x7f5dac0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d900662f0 0x7f5d900687a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac0d13f0 0x7f5dac152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 --2- 172.21.15.22:0/1163596223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac0b7ba0 0x7f5dac15eb70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.362+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 >> 172.21.15.22:0/1163596223 conn(0x7f5dac069960 msgr2=0x7f5dac0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:37.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.366+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 shutdown_connections 2023-06-23T18:03:37.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.366+0000 7f5db1402700 1 -- 172.21.15.22:0/1163596223 wait complete. 2023-06-23T18:03:37.379 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2023-06-23T18:03:37.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T18:03:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.498+0000 7f71a359e700 1 Processor -- start 2023-06-23T18:03:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.498+0000 7f71a359e700 1 -- start start 2023-06-23T18:03:37.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cc430 0x7f71a40b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f71a406c150 0x7f71a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a406cae0 con 0x7f71a40cd870 2023-06-23T18:03:37.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a40cc800 con 0x7f71a406c150 2023-06-23T18:03:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a406f9f0 con 0x7f71a40cc430 2023-06-23T18:03:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f71a406c150 0x7f71a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f71a406c150 0x7f71a406c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40238/0 (socket says 172.21.15.22:40238) 2023-06-23T18:03:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 -- 172.21.15.22:0/1678578949 learned_addr learned my addr 172.21.15.22:0/1678578949 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:37.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a259c700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cc430 0x7f71a40b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 -- 172.21.15.22:0/1678578949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cc430 msgr2=0x7f71a40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:37.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cc430 0x7f71a40b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 -- 172.21.15.22:0/1678578949 >> v1:172.21.15.169:6789/0 conn(0x7f71a40cd870 legacy=0x7f71a406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 -- 172.21.15.22:0/1678578949 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a415fa00 con 0x7f71a406c150 2023-06-23T18:03:37.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a2d9d700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a406c150 0x7f71a406c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f719c009e50 tx=0x7f719c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52935ebc4815334a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a1d9b700 1 -- 172.21.15.22:0/1678578949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00e070 con 0x7f71a406c150 2023-06-23T18:03:37.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a1d9b700 1 -- 172.21.15.22:0/1678578949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00b9c0 con 0x7f71a406c150 2023-06-23T18:03:37.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a1d9b700 1 -- 172.21.15.22:0/1678578949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00bd70 con 0x7f71a406c150 2023-06-23T18:03:37.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- 172.21.15.22:0/1678578949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a406c150 msgr2=0x7f71a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a406c150 0x7f71a406c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f719c009e50 tx=0x7f719c00b330 comp rx=0 tx=0).stop 2023-06-23T18:03:37.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- 172.21.15.22:0/1678578949 shutdown_connections 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a406c150 0x7f71a406c5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 --2- 172.21.15.22:0/1678578949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cc430 0x7f71a40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- 172.21.15.22:0/1678578949 >> 172.21.15.22:0/1678578949 conn(0x7f71a40ccd60 msgr2=0x7f71a40cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- 172.21.15.22:0/1678578949 shutdown_connections 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.502+0000 7f71a359e700 1 -- 172.21.15.22:0/1678578949 wait complete. 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 Processor -- start 2023-06-23T18:03:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 -- start start 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a406c150 0x7f71a415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cd870 0x7f71a414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40c8f40 con 0x7f71a40cc430 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40c5770 con 0x7f71a406c150 2023-06-23T18:03:37.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.506+0000 7f71a359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40c91e0 con 0x7f71a40cd870 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40246/0 (socket says 172.21.15.22:40246) 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 -- 172.21.15.22:0/118784447 learned_addr learned my addr 172.21.15.22:0/118784447 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f71a2d9d700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cd870 0x7f71a414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f71a259c700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a406c150 0x7f71a415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 -- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cd870 msgr2=0x7f71a414ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cd870 0x7f71a414ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 -- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a406c150 msgr2=0x7f71a415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a406c150 0x7f71a415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 -- 172.21.15.22:0/118784447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009b30 con 0x7f71a40cc430 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f7199d9b700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c500 tx=0x7f718c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c0052d0 con 0x7f71a40cc430 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c005430 con 0x7f71a40cc430 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a40c5a10 con 0x7f71a40cc430 2023-06-23T18:03:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c0055a0 con 0x7f71a40cc430 2023-06-23T18:03:37.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a415ba20 con 0x7f71a40cc430 2023-06-23T18:03:37.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.510+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a415bc10 con 0x7f71a40cc430 2023-06-23T18:03:37.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.514+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f718c0038e0 con 0x7f71a40cc430 2023-06-23T18:03:37.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.514+0000 7f719b7fe700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 0x7f71840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.514+0000 7f71a259c700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 0x7f71840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.514+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f718c082db0 con 0x7f71a40cc430 2023-06-23T18:03:37.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.514+0000 7f71a259c700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 0x7f71840687a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7194009a00 tx=0x7f7194008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.522+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f718c050320 con 0x7f71a40cc430 2023-06-23T18:03:37.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.722+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f71a40c55d0 con 0x7f71a40cc430 2023-06-23T18:03:37.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.738+0000 7f719b7fe700 1 -- 172.21.15.22:0/118784447 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f71a40c55d0 con 0x7f71a40cc430 2023-06-23T18:03:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 msgr2=0x7f71840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 0x7f71840687a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7194009a00 tx=0x7f7194008040 comp rx=0 tx=0).stop 2023-06-23T18:03:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 msgr2=0x7f71a415ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c500 tx=0x7f718c00c810 comp rx=0 tx=0).stop 2023-06-23T18:03:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 shutdown_connections 2023-06-23T18:03:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a40cd870 0x7f71a414ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71a40cc430 0x7f71a415ed40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f71840662f0 0x7f71840687a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 --2- 172.21.15.22:0/118784447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a406c150 0x7f71a415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 >> 172.21.15.22:0/118784447 conn(0x7f71a40ccd60 msgr2=0x7f71a40b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:37.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 shutdown_connections 2023-06-23T18:03:37.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.742+0000 7f71a359e700 1 -- 172.21.15.22:0/118784447 wait complete. 2023-06-23T18:03:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 3 and data pool 4 2023-06-23T18:03:37.772 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-06-23T18:03:37.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-06-23T18:03:37.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.878+0000 7fd6f814c700 1 Processor -- start 2023-06-23T18:03:37.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.878+0000 7fd6f814c700 1 -- start start 2023-06-23T18:03:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 0x7fd6f006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f006bcf0 con 0x7fd6f00d1d90 2023-06-23T18:03:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00d01e0 con 0x7fd6f015bdc0 2023-06-23T18:03:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00d2210 con 0x7fd6f006b350 2023-06-23T18:03:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f56e7700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd6f00d1d90 0x7fd6f015cb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45028/0 (socket says 172.21.15.22:45028) 2023-06-23T18:03:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f56e7700 1 -- 172.21.15.22:0/450875359 learned_addr learned my addr 172.21.15.22:0/450875359 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f66e9700 1 --2- 172.21.15.22:0/450875359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 0x7fd6f006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f4ee6700 1 -- 172.21.15.22:0/450875359 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624207973 0 0) 0x7fd6f006bcf0 con 0x7fd6f00d1d90 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f4ee6700 1 -- 172.21.15.22:0/450875359 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc0035d0 con 0x7fd6f00d1d90 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f4ee6700 1 -- 172.21.15.22:0/450875359 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907286187 0 0) 0x7fd6f00d01e0 con 0x7fd6f015bdc0 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f66e9700 1 -- 172.21.15.22:0/450875359 >> v1:172.21.15.169:6789/0 conn(0x7fd6f015bdc0 legacy=0x7fd6f0160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f66e9700 1 -- 172.21.15.22:0/450875359 >> v1:172.21.15.22:6789/0 conn(0x7fd6f00d1d90 legacy=0x7fd6f015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f66e9700 1 -- 172.21.15.22:0/450875359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00d04c0 con 0x7fd6f006b350 2023-06-23T18:03:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f66e9700 1 --2- 172.21.15.22:0/450875359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 0x7fd6f006b7b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009dc0 tx=0x7fd6e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26611256c4723b2b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f4ee6700 1 -- 172.21.15.22:0/450875359 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800e070 con 0x7fd6f006b350 2023-06-23T18:03:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f4ee6700 1 -- 172.21.15.22:0/450875359 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800bb20 con 0x7fd6f006b350 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- 172.21.15.22:0/450875359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 msgr2=0x7fd6f006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 --2- 172.21.15.22:0/450875359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 0x7fd6f006b7b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009dc0 tx=0x7fd6e800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- 172.21.15.22:0/450875359 shutdown_connections 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 --2- 172.21.15.22:0/450875359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f006b350 0x7fd6f006b7b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.882+0000 7fd6f814c700 1 -- 172.21.15.22:0/450875359 >> 172.21.15.22:0/450875359 conn(0x7fd6f00d3200 msgr2=0x7fd6f00ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- 172.21.15.22:0/450875359 shutdown_connections 2023-06-23T18:03:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- 172.21.15.22:0/450875359 wait complete. 2023-06-23T18:03:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 Processor -- start 2023-06-23T18:03:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- start start 2023-06-23T18:03:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 0x7fd6f00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 0x7fd6f00ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f014bb70 con 0x7fd6f006b350 2023-06-23T18:03:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00b71d0 con 0x7fd6f015bdc0 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.886+0000 7fd6f814c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00b74b0 con 0x7fd6f00d1d90 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40258/0 (socket says 172.21.15.22:40258) 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f66e9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 0x7fd6f00ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 -- 172.21.15.22:0/2901128116 learned_addr learned my addr 172.21.15.22:0/2901128116 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f56e7700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 0x7fd6f00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 -- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 msgr2=0x7fd6f00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 0x7fd6f00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 -- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 msgr2=0x7fd6f00ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 0x7fd6f00ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 -- 172.21.15.22:0/2901128116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009a70 con 0x7fd6f006b350 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f66e9700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 0x7fd6f00ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f56e7700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 0x7fd6f00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f5ee8700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c520 tx=0x7fd6ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0054e0 con 0x7fd6f006b350 2023-06-23T18:03:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec0036a0 con 0x7fd6f006b350 2023-06-23T18:03:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00c7b20 con 0x7fd6f006b350 2023-06-23T18:03:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec003a10 con 0x7fd6f006b350 2023-06-23T18:03:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.890+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f00c7810 con 0x7fd6f006b350 2023-06-23T18:03:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f004dc70 con 0x7fd6f006b350 2023-06-23T18:03:37.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd6ec003b70 con 0x7fd6f006b350 2023-06-23T18:03:37.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6e67fc700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 0x7fd6d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:37.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6ec083090 con 0x7fd6f006b350 2023-06-23T18:03:37.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6f56e7700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 0x7fd6d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.894+0000 7fd6f56e7700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 0x7fd6d40687a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0007fd0 tx=0x7fd6e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:37.902+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd6ec050520 con 0x7fd6f006b350 2023-06-23T18:03:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.078+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fd6f00cdfd0 con 0x7fd6f006b350 2023-06-23T18:03:38.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.745+0000 7fd6e67fc700 1 -- 172.21.15.22:0/2901128116 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fd6ec0170e0 con 0x7fd6f006b350 2023-06-23T18:03:38.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 msgr2=0x7fd6d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:38.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 0x7fd6d40687a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0007fd0 tx=0x7fd6e0009040 comp rx=0 tx=0).stop 2023-06-23T18:03:38.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 msgr2=0x7fd6f01457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:38.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c520 tx=0x7fd6ec00c830 comp rx=0 tx=0).stop 2023-06-23T18:03:38.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 shutdown_connections 2023-06-23T18:03:38.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6d40662f0 0x7fd6d40687a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f015bdc0 0x7fd6f00ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f00d1d90 0x7fd6f00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 --2- 172.21.15.22:0/2901128116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6f006b350 0x7fd6f01457f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 >> 172.21.15.22:0/2901128116 conn(0x7fd6f00d3200 msgr2=0x7fd6f0159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 shutdown_connections 2023-06-23T18:03:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.749+0000 7fd6f814c700 1 -- 172.21.15.22:0/2901128116 wait complete. 2023-06-23T18:03:38.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T18:03:38.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.901+0000 7f783e2b3700 1 Processor -- start 2023-06-23T18:03:38.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.901+0000 7f783e2b3700 1 -- start start 2023-06-23T18:03:38.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 0x7f78380ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78380cf2e0 0x7f783815f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380cca60 con 0x7f78380cf2e0 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380d1390 con 0x7f78380d0520 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380d0970 con 0x7f783815f410 2023-06-23T18:03:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78380cf2e0 0x7f783815f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78380cf2e0 0x7f783815f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40264/0 (socket says 172.21.15.22:40264) 2023-06-23T18:03:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 -- 172.21.15.22:0/2897142406 learned_addr learned my addr 172.21.15.22:0/2897142406 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783cab0700 1 --2- 172.21.15.22:0/2897142406 >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 0x7f78380ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783d2b1700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 -- 172.21.15.22:0/2897142406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 msgr2=0x7f78380cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 -- 172.21.15.22:0/2897142406 >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 msgr2=0x7f78380ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 --2- 172.21.15.22:0/2897142406 >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 0x7f78380ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 -- 172.21.15.22:0/2897142406 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380cfb10 con 0x7f78380cf2e0 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783cab0700 1 --2- 172.21.15.22:0/2897142406 >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 0x7f78380ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783d2b1700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783dab2700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f783815f8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78380612b0 tx=0x7f783000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb401067e8edc9eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f782ffff700 1 -- 172.21.15.22:0/2897142406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000e070 con 0x7f78380cf2e0 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f782ffff700 1 -- 172.21.15.22:0/2897142406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783000b910 con 0x7f78380cf2e0 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f782ffff700 1 -- 172.21.15.22:0/2897142406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000bcc0 con 0x7f78380cf2e0 2023-06-23T18:03:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 -- 172.21.15.22:0/2897142406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 msgr2=0x7f783815f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f783815f8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78380612b0 tx=0x7f783000b330 comp rx=0 tx=0).stop 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 -- 172.21.15.22:0/2897142406 shutdown_connections 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f783815f8c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- 172.21.15.22:0/2897142406 >> v2:172.21.15.169:3300/0 conn(0x7f78380d0520 0x7f78380ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.905+0000 7f783e2b3700 1 --2- 172.21.15.22:0/2897142406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.909+0000 7f783e2b3700 1 -- 172.21.15.22:0/2897142406 >> 172.21.15.22:0/2897142406 conn(0x7f783809b770 msgr2=0x7f783806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.909+0000 7f783e2b3700 1 -- 172.21.15.22:0/2897142406 shutdown_connections 2023-06-23T18:03:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.909+0000 7f783e2b3700 1 -- 172.21.15.22:0/2897142406 wait complete. 2023-06-23T18:03:38.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.909+0000 7f783e2b3700 1 Processor -- start 2023-06-23T18:03:38.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.909+0000 7f783e2b3700 1 -- start start 2023-06-23T18:03:38.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78380d0520 0x7f78381f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78381f5630 con 0x7f78380cf2e0 2023-06-23T18:03:38.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380b1b90 con 0x7f78380d0520 2023-06-23T18:03:38.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380c6ce0 con 0x7f783815f410 2023-06-23T18:03:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40272/0 (socket says 172.21.15.22:40272) 2023-06-23T18:03:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 -- 172.21.15.22:0/3083708009 learned_addr learned my addr 172.21.15.22:0/3083708009 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783dab2700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783cab0700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78380d0520 0x7f78381f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 -- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 msgr2=0x7f78380c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 -- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78380d0520 msgr2=0x7f78381f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78380d0520 0x7f78381f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 -- 172.21.15.22:0/3083708009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340094b0 con 0x7f78380cf2e0 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783d2b1700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f782800ec10 tx=0x7f782800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800c850 con 0x7f78380cf2e0 2023-06-23T18:03:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78280094b0 con 0x7f78380cf2e0 2023-06-23T18:03:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7830009b30 con 0x7f78380cf2e0 2023-06-23T18:03:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800c9b0 con 0x7f78380cf2e0 2023-06-23T18:03:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.913+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800cd40 con 0x7f78380cf2e0 2023-06-23T18:03:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.917+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7818005310 con 0x7f78380cf2e0 2023-06-23T18:03:38.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.917+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7828013470 con 0x7f78380cf2e0 2023-06-23T18:03:38.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.917+0000 7f782dffb700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 0x7f781c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:38.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.917+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f7828083e30 con 0x7f78380cf2e0 2023-06-23T18:03:38.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.917+0000 7f783cab0700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 0x7f781c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:38.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.921+0000 7f783cab0700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 0x7f781c0687a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7838159260 tx=0x7f7834009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:38.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:38.925+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7828054c70 con 0x7f78380cf2e0 2023-06-23T18:03:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:39.089+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f78180055c0 con 0x7f78380cf2e0 2023-06-23T18:03:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.753+0000 7f782dffb700 1 -- 172.21.15.22:0/3083708009 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f782800cea0 con 0x7f78380cf2e0 2023-06-23T18:03:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 msgr2=0x7f781c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 0x7f781c0687a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7838159260 tx=0x7f7834009240 comp rx=0 tx=0).stop 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 msgr2=0x7f78380c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f782800ec10 tx=0x7f782800ef20 comp rx=0 tx=0).stop 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 shutdown_connections 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783815f410 0x7f78380c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f781c0662f0 0x7f781c0687a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78380d0520 0x7f78381f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 --2- 172.21.15.22:0/3083708009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf2e0 0x7f78380c67a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 >> 172.21.15.22:0/3083708009 conn(0x7f783809b770 msgr2=0x7f78380ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 shutdown_connections 2023-06-23T18:03:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.769+0000 7f783e2b3700 1 -- 172.21.15.22:0/3083708009 wait complete. 2023-06-23T18:03:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' created 2023-06-23T18:03:40.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-06-23T18:03:40.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.905+0000 7f1d0a383700 1 Processor -- start 2023-06-23T18:03:40.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.905+0000 7f1d0a383700 1 -- start start 2023-06-23T18:03:40.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d04070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d040b01f0 0x7f1d040c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040c8750 con 0x7f1d040cc5d0 2023-06-23T18:03:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d04070670 con 0x7f1d040b01f0 2023-06-23T18:03:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040ccc20 con 0x7f1d041636c0 2023-06-23T18:03:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d08b80700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d040cc5d0 0x7f1d040afae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45034/0 (socket says 172.21.15.22:45034) 2023-06-23T18:03:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d08b80700 1 -- 172.21.15.22:0/2975413648 learned_addr learned my addr 172.21.15.22:0/2975413648 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:40.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09b82700 1 --2- 172.21.15.22:0/2975413648 >> v2:172.21.15.169:3300/0 conn(0x7f1d040b01f0 0x7f1d040c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 --2- 172.21.15.22:0/2975413648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d04070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848982326 0 0) 0x7f1d040c8750 con 0x7f1d040cc5d0 2023-06-23T18:03:40.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec0035d0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2946687338 0 0) 0x7f1cec0035d0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 -- 172.21.15.22:0/2975413648 >> v2:172.21.15.169:3300/0 conn(0x7f1d040b01f0 msgr2=0x7f1d040c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:40.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 --2- 172.21.15.22:0/2975413648 >> v2:172.21.15.169:3300/0 conn(0x7f1d040b01f0 0x7f1d040c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 -- 172.21.15.22:0/2975413648 >> v1:172.21.15.22:6789/0 conn(0x7f1d040cc5d0 legacy=0x7f1d040afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 -- 172.21.15.22:0/2975413648 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040ccf00 con 0x7f1d041636c0 2023-06-23T18:03:40.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d09381700 1 --2- 172.21.15.22:0/2975413648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d04070130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009d90 tx=0x7f1cf400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad210623f1b8123 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:40.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400e070 con 0x7f1d041636c0 2023-06-23T18:03:40.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4003a40 con 0x7f1d041636c0 2023-06-23T18:03:40.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1cfbfff700 1 -- 172.21.15.22:0/2975413648 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00001670 con 0x7f1d041636c0 2023-06-23T18:03:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- 172.21.15.22:0/2975413648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 msgr2=0x7f1d04070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 --2- 172.21.15.22:0/2975413648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d04070130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009d90 tx=0x7f1cf400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- 172.21.15.22:0/2975413648 shutdown_connections 2023-06-23T18:03:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 --2- 172.21.15.22:0/2975413648 >> v2:172.21.15.169:3300/0 conn(0x7f1d040b01f0 0x7f1d040c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 --2- 172.21.15.22:0/2975413648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d04070130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- 172.21.15.22:0/2975413648 >> 172.21.15.22:0/2975413648 conn(0x7f1d040ce060 msgr2=0x7f1d040ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- 172.21.15.22:0/2975413648 shutdown_connections 2023-06-23T18:03:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.909+0000 7f1d0a383700 1 -- 172.21.15.22:0/2975413648 wait complete. 2023-06-23T18:03:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 Processor -- start 2023-06-23T18:03:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 -- start start 2023-06-23T18:03:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040b01f0 0x7f1d040aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d040c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0415fae0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b3d30 con 0x7f1d040b01f0 2023-06-23T18:03:40.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d0a383700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040c47f0 con 0x7f1d041636c0 2023-06-23T18:03:40.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40282/0 (socket says 172.21.15.22:40282) 2023-06-23T18:03:40.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 -- 172.21.15.22:0/1249974372 learned_addr learned my addr 172.21.15.22:0/1249974372 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:40.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d09b82700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d040c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d09381700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040b01f0 0x7f1d040aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 -- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 msgr2=0x7f1d040c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d040c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 -- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040b01f0 msgr2=0x7f1d040aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:40.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040b01f0 0x7f1d040aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 -- 172.21.15.22:0/1249974372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009a70 con 0x7f1d040cc5d0 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.913+0000 7f1d08b80700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c7b0 tx=0x7f1d0000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004ca0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00004e00 con 0x7f1d040cc5d0 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d040af220 con 0x7f1d040cc5d0 2023-06-23T18:03:40.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00007310 con 0x7f1d040cc5d0 2023-06-23T18:03:40.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040af7e0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d040cef70 con 0x7f1d040cc5d0 2023-06-23T18:03:40.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.917+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1d00007470 con 0x7f1d040cc5d0 2023-06-23T18:03:40.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.921+0000 7f1cf9ffb700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 0x7f1cf00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:40.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.921+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f1d00083dd0 con 0x7f1d040cc5d0 2023-06-23T18:03:40.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.921+0000 7f1d09381700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 0x7f1cf00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:40.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.921+0000 7f1d09381700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 0x7f1cf00687d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400b370 tx=0x7f1cf4011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:40.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:40.925+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1d00050ee0 con 0x7f1d040cc5d0 2023-06-23T18:03:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.089+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1d0404e4c0 con 0x7f1d040cc5d0 2023-06-23T18:03:41.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.789+0000 7f1cf9ffb700 1 -- 172.21.15.22:0/1249974372 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1d00007720 con 0x7f1d040cc5d0 2023-06-23T18:03:41.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 msgr2=0x7f1cf00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:41.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 0x7f1cf00687d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400b370 tx=0x7f1cf4011000 comp rx=0 tx=0).stop 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 msgr2=0x7f1d040b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c7b0 tx=0x7f1d0000cac0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 shutdown_connections 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d041636c0 0x7f1d040c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d040cc5d0 0x7f1d040b37f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cf0066320 0x7f1cf00687d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 --2- 172.21.15.22:0/1249974372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040b01f0 0x7f1d040aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 >> 172.21.15.22:0/1249974372 conn(0x7f1d040ce060 msgr2=0x7f1d040ac460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:41.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 shutdown_connections 2023-06-23T18:03:41.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.793+0000 7f1d0a383700 1 -- 172.21.15.22:0/1249974372 wait complete. 2023-06-23T18:03:41.808 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 5 allow_ec_overwrites to true 2023-06-23T18:03:41.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-06-23T18:03:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.921+0000 7f4d6a0b7700 1 Processor -- start 2023-06-23T18:03:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.921+0000 7f4d6a0b7700 1 -- start start 2023-06-23T18:03:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d640cf3a0 0x7f4d640cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 0x7f4d640d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4d640d2360 0x7f4d640d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640ccb30 con 0x7f4d640d2360 2023-06-23T18:03:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640cfcb0 con 0x7f4d640cf3a0 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640d2880 con 0x7f4d640d1f30 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4d640d2360 0x7f4d640d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4d640d2360 0x7f4d640d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40286/0 (socket says 172.21.15.22:40286) 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 -- 172.21.15.22:0/4072244721 learned_addr learned my addr 172.21.15.22:0/4072244721 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d690b5700 1 --2- 172.21.15.22:0/4072244721 >> v2:172.21.15.169:3300/0 conn(0x7f4d640cf3a0 0x7f4d640cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d688b4700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 0x7f4d640d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 -- 172.21.15.22:0/4072244721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 msgr2=0x7f4d640d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:41.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 0x7f4d640d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 -- 172.21.15.22:0/4072244721 >> v2:172.21.15.169:3300/0 conn(0x7f4d640cf3a0 msgr2=0x7f4d640cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 --2- 172.21.15.22:0/4072244721 >> v2:172.21.15.169:3300/0 conn(0x7f4d640cf3a0 0x7f4d640cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 -- 172.21.15.22:0/4072244721 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640d2b60 con 0x7f4d640d2360 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d688b4700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 0x7f4d640d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d698b6700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d640d0560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4d6408c5c0 tx=0x7f4d54009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a2f235c30f21a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d5bfff700 1 -- 172.21.15.22:0/4072244721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011830 con 0x7f4d640d2360 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d5bfff700 1 -- 172.21.15.22:0/4072244721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54011990 con 0x7f4d640d2360 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d5bfff700 1 -- 172.21.15.22:0/4072244721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011d20 con 0x7f4d640d2360 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/4072244721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 msgr2=0x7f4d640d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d640d0560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4d6408c5c0 tx=0x7f4d54009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/4072244721 shutdown_connections 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d640d0560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/4072244721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640d1f30 0x7f4d640d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/4072244721 >> v2:172.21.15.169:3300/0 conn(0x7f4d640cf3a0 0x7f4d640cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/4072244721 >> 172.21.15.22:0/4072244721 conn(0x7f4d6406a280 msgr2=0x7f4d6409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/4072244721 shutdown_connections 2023-06-23T18:03:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.925+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/4072244721 wait complete. 2023-06-23T18:03:41.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 Processor -- start 2023-06-23T18:03:41.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 -- start start 2023-06-23T18:03:41.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 0x7f4d641f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 0x7f4d640b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415eff0 con 0x7f4d640d2360 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641f15c0 con 0x7f4d640d1f30 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d6a0b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640b7fa0 con 0x7f4d640cf3a0 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d688b4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 0x7f4d640b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d690b5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 0x7f4d641f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40302/0 (socket says 172.21.15.22:40302) 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 -- 172.21.15.22:0/2580013114 learned_addr learned my addr 172.21.15.22:0/2580013114 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 -- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 msgr2=0x7f4d641f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 0x7f4d641f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 -- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 msgr2=0x7f4d640b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.929+0000 7f4d698b6700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 0x7f4d640b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d698b6700 1 -- 172.21.15.22:0/2580013114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009570 con 0x7f4d640d2360 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d690b5700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 0x7f4d641f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d688b4700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 0x7f4d640b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d698b6700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d54014040 tx=0x7f4d54004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54004460 con 0x7f4d640d2360 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5401e070 con 0x7f4d640d2360 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6415e650 con 0x7f4d640d2360 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54003900 con 0x7f4d640d2360 2023-06-23T18:03:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6415ec10 con 0x7f4d640d2360 2023-06-23T18:03:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640c8b00 con 0x7f4d640d2360 2023-06-23T18:03:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.933+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4d54003a60 con 0x7f4d640d2360 2023-06-23T18:03:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.937+0000 7f4d59ffb700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 0x7f4d500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.937+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f4d54020070 con 0x7f4d640d2360 2023-06-23T18:03:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.937+0000 7f4d690b5700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 0x7f4d500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.937+0000 7f4d690b5700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 0x7f4d500687a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009f50 tx=0x7f4d5c0091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:41.941+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4d640c8b00 con 0x7f4d640d2360 2023-06-23T18:03:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.141+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f4d640c8b00 con 0x7f4d640d2360 2023-06-23T18:03:42.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.149+0000 7f4d59ffb700 1 -- 172.21.15.22:0/2580013114 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f4d640c8b00 con 0x7f4d640d2360 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 msgr2=0x7f4d500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 0x7f4d500687a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009f50 tx=0x7f4d5c0091c0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 msgr2=0x7f4d6415d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d54014040 tx=0x7f4d54004770 comp rx=0 tx=0).stop 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 shutdown_connections 2023-06-23T18:03:42.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d640d2360 0x7f4d6415d8a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4d500662f0 0x7f4d500687a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d640d1f30 0x7f4d640b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 --2- 172.21.15.22:0/2580013114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d640cf3a0 0x7f4d641f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 >> 172.21.15.22:0/2580013114 conn(0x7f4d6406a280 msgr2=0x7f4d6406b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 shutdown_connections 2023-06-23T18:03:42.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.153+0000 7f4d6a0b7700 1 -- 172.21.15.22:0/2580013114 wait complete. 2023-06-23T18:03:42.167 INFO:teuthology.orchestra.run.smithi022.stderr:added data pool 5 to fsmap 2023-06-23T18:03:42.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:03:42.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 Processor -- start 2023-06-23T18:03:42.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 -- start start 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f902015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200d04a0 con 0x7f902015f9b0 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200d0a60 con 0x7f90200cd0d0 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f9024899700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cd4a0 con 0x7f90200b5920 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901e59c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f902015f9b0 0x7f90200b5210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45050/0 (socket says 172.21.15.22:45050) 2023-06-23T18:03:42.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901e59c700 1 -- 172.21.15.22:0/3645222007 learned_addr learned my addr 172.21.15.22:0/3645222007 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:42.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901f59e700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f902015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759670916 0 0) 0x7f90200d04a0 con 0x7f902015f9b0 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90080035d0 con 0x7f902015f9b0 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606308829 0 0) 0x7f90200d0a60 con 0x7f90200cd0d0 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90200d04a0 con 0x7f90200cd0d0 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4281602978 0 0) 0x7f90080035d0 con 0x7f902015f9b0 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 msgr2=0x7f902015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f902015fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 >> v1:172.21.15.169:6789/0 conn(0x7f90200cd0d0 legacy=0x7f90200d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020160470 con 0x7f902015f9b0 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f90100019f0 con 0x7f902015f9b0 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 >> v1:172.21.15.22:6789/0 conn(0x7f902015f9b0 legacy=0x7f90200b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008003be0 0x7f9008003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f9008004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9008004db0 0x7f9008005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cd4a0 con 0x7f9008003be0 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900800bb30 con 0x7f9008004db0 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900800bcf0 con 0x7f90200b5920 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901ed9d700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008003be0 0x7f9008003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901e59c700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f9008004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901f59e700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9008004db0 0x7f9008005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.277+0000 7f901ed9d700 1 -- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 msgr2=0x7f9008004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901ed9d700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90200b5920 0x7f9008004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901ed9d700 1 -- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9008004db0 msgr2=0x7f9008005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901ed9d700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9008004db0 0x7f9008005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901ed9d700 1 -- 172.21.15.22:0/3645222007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080035d0 con 0x7f9008003be0 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901f59e700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9008004db0 0x7f9008005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901ed9d700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008003be0 0x7f9008003fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9014001960 tx=0x7f90080058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34390c4d49c9efc5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014001450 con 0x7f9008003be0 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90140052f0 con 0x7f9008003be0 2023-06-23T18:03:42.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901dd9b700 1 -- 172.21.15.22:0/3645222007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140036a0 con 0x7f9008003be0 2023-06-23T18:03:42.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- 172.21.15.22:0/3645222007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008003be0 msgr2=0x7f9008003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 --2- 172.21.15.22:0/3645222007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9008003be0 0x7f9008003fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9014001960 tx=0x7f90080058a0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f901f59e700 1 -- 172.21.15.22:0/3645222007 reap_dead start 2023-06-23T18:03:42.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- 172.21.15.22:0/3645222007 shutdown_connections 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- 172.21.15.22:0/3645222007 >> 172.21.15.22:0/3645222007 conn(0x7f9020069960 msgr2=0x7f9020069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- 172.21.15.22:0/3645222007 shutdown_connections 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- 172.21.15.22:0/3645222007 wait complete. 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 Processor -- start 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.281+0000 7f9024899700 1 -- start start 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9008003be0 0x7f9020158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902015e760 0x7f902015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902015f110 con 0x7f90200cb0f0 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c7bb0 con 0x7f902015e760 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c7e50 con 0x7f9008003be0 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40316/0 (socket says 172.21.15.22:40316) 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 -- 172.21.15.22:0/9442069 learned_addr learned my addr 172.21.15.22:0/9442069 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901e59c700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902015e760 0x7f902015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901ed9d700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9008003be0 0x7f9020158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 -- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9008003be0 msgr2=0x7f9020158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9008003be0 0x7f9020158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 -- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902015e760 msgr2=0x7f902015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902015e760 0x7f902015ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 -- 172.21.15.22:0/9442069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200c8130 con 0x7f90200cb0f0 2023-06-23T18:03:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f901f59e700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90200d03c0 tx=0x7f9018011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018004230 con 0x7f90200cb0f0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90180043b0 con 0x7f90200cb0f0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90200c8410 con 0x7f90200cb0f0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90180036c0 con 0x7f90200cb0f0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.285+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90200c96c0 con 0x7f90200cb0f0 2023-06-23T18:03:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90200c8930 con 0x7f90200cb0f0 2023-06-23T18:03:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9018003820 con 0x7f90200cb0f0 2023-06-23T18:03:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f90077fe700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 0x7f900c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f901ed9d700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 0x7f900c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90180840d0 con 0x7f90200cb0f0 2023-06-23T18:03:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.289+0000 7f901ed9d700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 0x7f900c0687d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9014015040 tx=0x7f901400a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.297+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f90200c8930 con 0x7f90200cb0f0 2023-06-23T18:03:42.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.501+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90200c8930 con 0x7f90200cb0f0 2023-06-23T18:03:42.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.501+0000 7f90077fe700 1 -- 172.21.15.22:0/9442069 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f90200c8930 con 0x7f90200cb0f0 2023-06-23T18:03:42.515 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:42.516 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:03:42.145591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-06-23T18:03:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 msgr2=0x7f900c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 0x7f900c0687d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9014015040 tx=0x7f901400a000 comp rx=0 tx=0).stop 2023-06-23T18:03:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 msgr2=0x7f90200cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90200d03c0 tx=0x7f9018011770 comp rx=0 tx=0).stop 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 shutdown_connections 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90200cb0f0 0x7f90200cb590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f900c066320 0x7f900c0687d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f902015e760 0x7f902015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 --2- 172.21.15.22:0/9442069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9008003be0 0x7f9020158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 >> 172.21.15.22:0/9442069 conn(0x7f9020069960 msgr2=0x7f90200b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 shutdown_connections 2023-06-23T18:03:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.505+0000 7f9024899700 1 -- 172.21.15.22:0/9442069 wait complete. 2023-06-23T18:03:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-06-23T18:03:42.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:42.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 Processor -- start 2023-06-23T18:03:42.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 -- start start 2023-06-23T18:03:42.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16800b01a0 0x7f16800c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16800c93a0 0x7f168006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800c9b50 con 0x7f1680070280 2023-06-23T18:03:42.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16801590a0 con 0x7f16800b01a0 2023-06-23T18:03:42.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.633+0000 7f1684da9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800c9e30 con 0x7f16800c93a0 2023-06-23T18:03:42.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16800b01a0 0x7f16800c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16800b01a0 0x7f16800c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40318/0 (socket says 172.21.15.22:40318) 2023-06-23T18:03:42.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 -- 172.21.15.22:0/1691047824 learned_addr learned my addr 172.21.15.22:0/1691047824 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:42.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f167ffff700 1 --2- 172.21.15.22:0/1691047824 >> v2:172.21.15.169:3300/0 conn(0x7f16800c93a0 0x7f168006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 -- 172.21.15.22:0/1691047824 >> v2:172.21.15.169:3300/0 conn(0x7f16800c93a0 msgr2=0x7f168006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 --2- 172.21.15.22:0/1691047824 >> v2:172.21.15.169:3300/0 conn(0x7f16800c93a0 0x7f168006b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 -- 172.21.15.22:0/1691047824 >> v1:172.21.15.169:6789/0 conn(0x7f1680070280 legacy=0x7f16800afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 -- 172.21.15.22:0/1691047824 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16800ca110 con 0x7f16800b01a0 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1677fff700 1 --2- 172.21.15.22:0/1691047824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f16800c8e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f168006b170 tx=0x7f1668009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1562abe92c7e8596 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f167effd700 1 -- 172.21.15.22:0/1691047824 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011830 con 0x7f16800b01a0 2023-06-23T18:03:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f167effd700 1 -- 172.21.15.22:0/1691047824 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668011990 con 0x7f16800b01a0 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f167effd700 1 -- 172.21.15.22:0/1691047824 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011d20 con 0x7f16800b01a0 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 -- 172.21.15.22:0/1691047824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 msgr2=0x7f16800c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 --2- 172.21.15.22:0/1691047824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f16800c8e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f168006b170 tx=0x7f1668009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 -- 172.21.15.22:0/1691047824 shutdown_connections 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 --2- 172.21.15.22:0/1691047824 >> v2:172.21.15.169:3300/0 conn(0x7f16800c93a0 0x7f168006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 --2- 172.21.15.22:0/1691047824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f16800c8e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 -- 172.21.15.22:0/1691047824 >> 172.21.15.22:0/1691047824 conn(0x7f168006a5f0 msgr2=0x7f16800caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 -- 172.21.15.22:0/1691047824 shutdown_connections 2023-06-23T18:03:42.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.637+0000 7f1684da9700 1 -- 172.21.15.22:0/1691047824 wait complete. 2023-06-23T18:03:42.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 Processor -- start 2023-06-23T18:03:42.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- start start 2023-06-23T18:03:42.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1680070280 0x7f16800b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16800c93a0 0x7f16800b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680156210 con 0x7f16800b01a0 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800b21a0 con 0x7f1680070280 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800b1900 con 0x7f16800c93a0 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40332/0 (socket says 172.21.15.22:40332) 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 -- 172.21.15.22:0/985277699 learned_addr learned my addr 172.21.15.22:0/985277699 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f167f7fe700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1680070280 0x7f16800b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f167ffff700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16800c93a0 0x7f16800b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 -- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16800c93a0 msgr2=0x7f16800b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16800c93a0 0x7f16800b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 -- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1680070280 msgr2=0x7f16800b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1680070280 0x7f16800b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 -- 172.21.15.22:0/985277699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009570 con 0x7f16800b01a0 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1677fff700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f168006dc10 tx=0x7f1668004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668004450 con 0x7f16800b01a0 2023-06-23T18:03:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166801f070 con 0x7f16800b01a0 2023-06-23T18:03:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16800b1be0 con 0x7f16800b01a0 2023-06-23T18:03:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668003900 con 0x7f16800b01a0 2023-06-23T18:03:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.641+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168015c050 con 0x7f16800b01a0 2023-06-23T18:03:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1668003a60 con 0x7f16800b01a0 2023-06-23T18:03:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1680156380 con 0x7f16800b01a0 2023-06-23T18:03:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f167cff9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 0x7f16640687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f167f7fe700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 0x7f16640687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1668021070 con 0x7f16800b01a0 2023-06-23T18:03:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.645+0000 7f167f7fe700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 0x7f16640687d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f60 tx=0x7f1670009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.649+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f16680525e0 con 0x7f16800b01a0 2023-06-23T18:03:42.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.817+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f168004e4c0 con 0x7f16800b01a0 2023-06-23T18:03:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f167cff9700 1 -- 172.21.15.22:0/985277699 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f16680045d0 con 0x7f16800b01a0 2023-06-23T18:03:42.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:42.832 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 msgr2=0x7f16640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 0x7f16640687d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f60 tx=0x7f1670009040 comp rx=0 tx=0).stop 2023-06-23T18:03:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 msgr2=0x7f1680155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f168006dc10 tx=0x7f1668004770 comp rx=0 tx=0).stop 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 shutdown_connections 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16800c93a0 0x7f16800b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16800b01a0 0x7f1680155810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1664066320 0x7f16640687d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 --2- 172.21.15.22:0/985277699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1680070280 0x7f16800b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 >> 172.21.15.22:0/985277699 conn(0x7f168006a5f0 msgr2=0x7f16800cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 shutdown_connections 2023-06-23T18:03:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.821+0000 7f1684da9700 1 -- 172.21.15.22:0/985277699 wait complete. 2023-06-23T18:03:42.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 Processor -- start 2023-06-23T18:03:42.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 -- start start 2023-06-23T18:03:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 0x7f92b40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 0x7f92b40d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40b7ec0 con 0x7f92b40cc9e0 2023-06-23T18:03:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40b1ad0 con 0x7f92b40cf480 2023-06-23T18:03:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40d0a10 con 0x7f92b40b7af0 2023-06-23T18:03:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b27fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f92b40cc9e0 0x7f92b40ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45060/0 (socket says 172.21.15.22:45060) 2023-06-23T18:03:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b27fc700 1 -- 172.21.15.22:0/1139679582 learned_addr learned my addr 172.21.15.22:0/1139679582 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1139679582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 0x7f92b40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b37fe700 1 --2- 172.21.15.22:0/1139679582 >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 0x7f92b40d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b1ffb700 1 -- 172.21.15.22:0/1139679582 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746620737 0 0) 0x7f92b40b7ec0 con 0x7f92b40cc9e0 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b1ffb700 1 -- 172.21.15.22:0/1139679582 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f929c0035d0 con 0x7f92b40cc9e0 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1139679582 >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 msgr2=0x7f92b40d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1139679582 >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 0x7f92b40d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1139679582 >> v1:172.21.15.22:6789/0 conn(0x7f92b40cc9e0 legacy=0x7f92b40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1139679582 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40d0cb0 con 0x7f92b40b7af0 2023-06-23T18:03:42.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b37fe700 1 --2- 172.21.15.22:0/1139679582 >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 0x7f92b40d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.941+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1139679582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 0x7f92b40ce540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009d90 tx=0x7f92a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff143927155589 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b1ffb700 1 -- 172.21.15.22:0/1139679582 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400e070 con 0x7f92b40b7af0 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b1ffb700 1 -- 172.21.15.22:0/1139679582 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a400bb20 con 0x7f92b40b7af0 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b1ffb700 1 -- 172.21.15.22:0/1139679582 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400be90 con 0x7f92b40b7af0 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- 172.21.15.22:0/1139679582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 msgr2=0x7f92b40ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1139679582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 0x7f92b40ce540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009d90 tx=0x7f92a400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:42.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- 172.21.15.22:0/1139679582 shutdown_connections 2023-06-23T18:03:42.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1139679582 >> v2:172.21.15.169:3300/0 conn(0x7f92b40cf480 0x7f92b40d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1139679582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40b7af0 0x7f92b40ce540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- 172.21.15.22:0/1139679582 >> 172.21.15.22:0/1139679582 conn(0x7f92b4069960 msgr2=0x7f92b4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:42.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- 172.21.15.22:0/1139679582 shutdown_connections 2023-06-23T18:03:42.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- 172.21.15.22:0/1139679582 wait complete. 2023-06-23T18:03:42.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 Processor -- start 2023-06-23T18:03:42.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- start start 2023-06-23T18:03:42.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 0x7f92b4158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 0x7f92b4144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415ec10 con 0x7f92b40b7af0 2023-06-23T18:03:42.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415d630 con 0x7f92b40cf480 2023-06-23T18:03:42.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415d910 con 0x7f92b40cc9e0 2023-06-23T18:03:42.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40348/0 (socket says 172.21.15.22:40348) 2023-06-23T18:03:42.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1002818045 learned_addr learned my addr 172.21.15.22:0/1002818045 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:42.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b27fc700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 0x7f92b4158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b37fe700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 0x7f92b4144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 msgr2=0x7f92b4158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 0x7f92b4158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 msgr2=0x7f92b4144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 0x7f92b4144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 -- 172.21.15.22:0/1002818045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0094b0 con 0x7f92b40b7af0 2023-06-23T18:03:42.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b27fc700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 0x7f92b4158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:42.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b37fe700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 0x7f92b4144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:42.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.945+0000 7f92b2ffd700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b370 tx=0x7f92a4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4012840 con 0x7f92b40b7af0 2023-06-23T18:03:42.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a40129a0 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a4009a70 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4012d50 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b415ded0 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404e4c0 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f92a4026410 con 0x7f92b40b7af0 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 0x7f92a00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f92a40867a0 con 0x7f92b40b7af0 2023-06-23T18:03:42.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f92b27fc700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 0x7f92a00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:42.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.949+0000 7f92b27fc700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 0x7f92a00687a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cf190 tx=0x7f92a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:42.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:42.957+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f92a4053a70 con 0x7f92b40b7af0 2023-06-23T18:03:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.125+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92b40b4740 con 0x7f92b40b7af0 2023-06-23T18:03:43.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.125+0000 7f929b7fe700 1 -- 172.21.15.22:0/1002818045 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f92a4057090 con 0x7f92b40b7af0 2023-06-23T18:03:43.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:43.140 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 msgr2=0x7f92a00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 0x7f92a00687a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cf190 tx=0x7f92a8009040 comp rx=0 tx=0).stop 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 msgr2=0x7f92b415e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b370 tx=0x7f92a4004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 shutdown_connections 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f92a00662f0 0x7f92a00687a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92b40cf480 0x7f92b4144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92b40cc9e0 0x7f92b4158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 --2- 172.21.15.22:0/1002818045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b40b7af0 0x7f92b415e6d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 >> 172.21.15.22:0/1002818045 conn(0x7f92b4069960 msgr2=0x7f92b40b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 shutdown_connections 2023-06-23T18:03:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.129+0000 7f92b3fff700 1 -- 172.21.15.22:0/1002818045 wait complete. 2023-06-23T18:03:43.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:03:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.289+0000 7fb2ac802700 1 Processor -- start 2023-06-23T18:03:43.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.289+0000 7fb2ac802700 1 -- start start 2023-06-23T18:03:43.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2a815c490 0x7fb2a815c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a815d430 con 0x7fb2a80cb830 2023-06-23T18:03:43.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a815d150 con 0x7fb2a815c490 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80cd100 con 0x7fb2a80cb460 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a759e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb2a80cb830 0x7fb2a80cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45064/0 (socket says 172.21.15.22:45064) 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a759e700 1 -- 172.21.15.22:0/1847899630 learned_addr learned my addr 172.21.15.22:0/1847899630 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a659c700 1 --2- 172.21.15.22:0/1847899630 >> v2:172.21.15.169:3300/0 conn(0x7fb2a815c490 0x7fb2a815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/1847899630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625479811 0 0) 0x7fb2a815d430 con 0x7fb2a80cb830 2023-06-23T18:03:43.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2940035d0 con 0x7fb2a80cb830 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400891030 0 0) 0x7fb2940035d0 con 0x7fb2a80cb830 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 -- 172.21.15.22:0/1847899630 >> v2:172.21.15.169:3300/0 conn(0x7fb2a815c490 msgr2=0x7fb2a815c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/1847899630 >> v2:172.21.15.169:3300/0 conn(0x7fb2a815c490 0x7fb2a815c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 -- 172.21.15.22:0/1847899630 >> v1:172.21.15.22:6789/0 conn(0x7fb2a80cb830 legacy=0x7fb2a80cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 -- 172.21.15.22:0/1847899630 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80cd3e0 con 0x7fb2a80cb460 2023-06-23T18:03:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/1847899630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80cdbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb298009850 tx=0x7fb298009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9513a2dfa407d790 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00019f0 con 0x7fb2a80cb460 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298004030 con 0x7fb2a80cb460 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2a5d9b700 1 -- 172.21.15.22:0/1847899630 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980043c0 con 0x7fb2a80cb460 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- 172.21.15.22:0/1847899630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 msgr2=0x7fb2a80cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 --2- 172.21.15.22:0/1847899630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80cdbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb298009850 tx=0x7fb298009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- 172.21.15.22:0/1847899630 shutdown_connections 2023-06-23T18:03:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 --2- 172.21.15.22:0/1847899630 >> v2:172.21.15.169:3300/0 conn(0x7fb2a815c490 0x7fb2a815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 --2- 172.21.15.22:0/1847899630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80cdbe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- 172.21.15.22:0/1847899630 >> 172.21.15.22:0/1847899630 conn(0x7fb2a806aa70 msgr2=0x7fb2a806ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:43.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- 172.21.15.22:0/1847899630 shutdown_connections 2023-06-23T18:03:43.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.293+0000 7fb2ac802700 1 -- 172.21.15.22:0/1847899630 wait complete. 2023-06-23T18:03:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 Processor -- start 2023-06-23T18:03:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 -- start start 2023-06-23T18:03:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c490 0x7fb2a80c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80b4f80 con 0x7fb2a80cb830 2023-06-23T18:03:43.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8152910 con 0x7fb2a815c490 2023-06-23T18:03:43.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2ac802700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8151d90 con 0x7fb2a80cb460 2023-06-23T18:03:43.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40354/0 (socket says 172.21.15.22:40354) 2023-06-23T18:03:43.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 -- 172.21.15.22:0/3636691052 learned_addr learned my addr 172.21.15.22:0/3636691052 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:43.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a759e700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c490 0x7fb2a80c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 -- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 msgr2=0x7fb2a80b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 -- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c490 msgr2=0x7fb2a80c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c490 0x7fb2a80c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 -- 172.21.15.22:0/3636691052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980094b0 con 0x7fb2a80cb830 2023-06-23T18:03:43.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb2a659c700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e840 tx=0x7fb29000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.297+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900095a0 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290013680 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a8152070 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900139f0 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a8152630 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a80cace0 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb290013b50 con 0x7fb2a80cb830 2023-06-23T18:03:43.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb29f7fe700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 0x7fb2880687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb290084600 con 0x7fb2a80cb830 2023-06-23T18:03:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.301+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 0x7fb2880687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.305+0000 7fb2a6d9d700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 0x7fb2880687d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.309+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb2900552b0 con 0x7fb2a80cb830 2023-06-23T18:03:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.505+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2a804e4c0 con 0x7fb2a80cb830 2023-06-23T18:03:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.505+0000 7fb29f7fe700 1 -- 172.21.15.22:0/3636691052 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fb290054e40 con 0x7fb2a80cb830 2023-06-23T18:03:43.517 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:43.518 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:03:42.145591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-06-23T18:03:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 msgr2=0x7fb2880687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 0x7fb2880687d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298008030 comp rx=0 tx=0).stop 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 msgr2=0x7fb2a815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e840 tx=0x7fb29000eb50 comp rx=0 tx=0).stop 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 shutdown_connections 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb288066320 0x7fb2880687d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c490 0x7fb2a80c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2a80cb830 0x7fb2a815b5d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 --2- 172.21.15.22:0/3636691052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a80cb460 0x7fb2a80b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 >> 172.21.15.22:0/3636691052 conn(0x7fb2a806aa70 msgr2=0x7fb2a80c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 shutdown_connections 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.509+0000 7fb2ac802700 1 -- 172.21.15.22:0/3636691052 wait complete. 2023-06-23T18:03:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-06-23T18:03:43.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:43.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 Processor -- start 2023-06-23T18:03:43.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 -- start start 2023-06-23T18:03:43.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9000acff0 0x7fe90006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe900071680 0x7fe9000affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90006d2c0 con 0x7fe900071680 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000cd1c0 con 0x7fe9000acff0 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.641+0000 7fe904cb6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000cd4a0 con 0x7fe9000ad3c0 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe900071680 0x7fe9000affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe900071680 0x7fe9000affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40358/0 (socket says 172.21.15.22:40358) 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 -- 172.21.15.22:0/337983 learned_addr learned my addr 172.21.15.22:0/337983 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8feffd700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8ff7fe700 1 --2- 172.21.15.22:0/337983 >> v2:172.21.15.169:3300/0 conn(0x7fe9000acff0 0x7fe90006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 -- 172.21.15.22:0/337983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 msgr2=0x7fe9000afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 -- 172.21.15.22:0/337983 >> v2:172.21.15.169:3300/0 conn(0x7fe9000acff0 msgr2=0x7fe90006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 --2- 172.21.15.22:0/337983 >> v2:172.21.15.169:3300/0 conn(0x7fe9000acff0 0x7fe90006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 -- 172.21.15.22:0/337983 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000cd780 con 0x7fe900071680 2023-06-23T18:03:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fffff700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe9000affd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe900061240 tx=0x7fe8f8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7455d8798ecbc8f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fe7fc700 1 -- 172.21.15.22:0/337983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800b8a0 con 0x7fe900071680 2023-06-23T18:03:43.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fe7fc700 1 -- 172.21.15.22:0/337983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f800ba00 con 0x7fe900071680 2023-06-23T18:03:43.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe8fe7fc700 1 -- 172.21.15.22:0/337983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800bd90 con 0x7fe900071680 2023-06-23T18:03:43.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 -- 172.21.15.22:0/337983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 msgr2=0x7fe9000affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe9000affd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe900061240 tx=0x7fe8f8009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 -- 172.21.15.22:0/337983 shutdown_connections 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe9000affd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 --2- 172.21.15.22:0/337983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 --2- 172.21.15.22:0/337983 >> v2:172.21.15.169:3300/0 conn(0x7fe9000acff0 0x7fe90006ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 -- 172.21.15.22:0/337983 >> 172.21.15.22:0/337983 conn(0x7fe90006a5f0 msgr2=0x7fe9000caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 -- 172.21.15.22:0/337983 shutdown_connections 2023-06-23T18:03:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.645+0000 7fe904cb6700 1 -- 172.21.15.22:0/337983 wait complete. 2023-06-23T18:03:43.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 Processor -- start 2023-06-23T18:03:43.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 -- start start 2023-06-23T18:03:43.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 0x7fe9001f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90014c970 con 0x7fe900071680 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000c32b0 con 0x7fe9000acff0 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe904cb6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000c25c0 con 0x7fe9000ad3c0 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40372/0 (socket says 172.21.15.22:40372) 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 -- 172.21.15.22:0/2095052489 learned_addr learned my addr 172.21.15.22:0/2095052489 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8feffd700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 0x7fe9001f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8fffff700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 -- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 msgr2=0x7fe9000c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.649+0000 7fe8ff7fe700 1 -- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 msgr2=0x7fe9001f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8ff7fe700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 0x7fe9001f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8ff7fe700 1 -- 172.21.15.22:0/2095052489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8009570 con 0x7fe900071680 2023-06-23T18:03:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8feffd700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 0x7fe9001f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8ff7fe700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e810 tx=0x7fe8f400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4013610 con 0x7fe900071680 2023-06-23T18:03:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4013790 con 0x7fe900071680 2023-06-23T18:03:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4013b40 con 0x7fe900071680 2023-06-23T18:03:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9000c28a0 con 0x7fe900071680 2023-06-23T18:03:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9000c3850 con 0x7fe900071680 2023-06-23T18:03:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9000c3a10 con 0x7fe900071680 2023-06-23T18:03:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe8f4013ca0 con 0x7fe900071680 2023-06-23T18:03:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8feffd700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.653+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe8f4083cf0 con 0x7fe900071680 2023-06-23T18:03:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.657+0000 7fe8feffd700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b3cc0 tx=0x7fe8f0003aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:43.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.661+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe8f4054870 con 0x7fe900071680 2023-06-23T18:03:43.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.825+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe90004e4c0 con 0x7fe900071680 2023-06-23T18:03:43.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.825+0000 7fe8e7fff700 1 -- 172.21.15.22:0/2095052489 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fe8f4009460 con 0x7fe900071680 2023-06-23T18:03:43.838 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:43.838 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:43.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 msgr2=0x7fe8ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b3cc0 tx=0x7fe8f0003aa0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 msgr2=0x7fe90014c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e810 tx=0x7fe8f400ebd0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 shutdown_connections 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9000ad3c0 0x7fe9000c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe8ec0662f0 0x7fe8ec0687a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9000acff0 0x7fe9001f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 --2- 172.21.15.22:0/2095052489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe900071680 0x7fe90014c430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:43.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 >> 172.21.15.22:0/2095052489 conn(0x7fe90006a5f0 msgr2=0x7fe9000cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:43.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 shutdown_connections 2023-06-23T18:03:43.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:43.829+0000 7fe904cb6700 1 -- 172.21.15.22:0/2095052489 wait complete. 2023-06-23T18:03:44.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-23T18:03:37.725874+0000', 'modified': '2023-06-23T18:03:42.145591+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6837/2486746630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6835', 'nonce': 2486746630}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 2486746630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-06-23T18:03:44.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-23T18:03:44.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-23T18:03:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.369+0000 7f35f7fff700 1 Processor -- start 2023-06-23T18:03:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.369+0000 7f35f7fff700 1 -- start start 2023-06-23T18:03:44.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f35f80b10b0 0x7f35f80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f80cf280 0x7f35f80c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f815c740 con 0x7f35f815c370 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f80d0fc0 con 0x7f35f80b10b0 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f80cf700 con 0x7f35f80cf280 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f35f80b10b0 0x7f35f80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f6ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f35f815c370 0x7f35f80d13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45076/0 (socket says 172.21.15.22:45076) 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f6ffd700 1 -- 172.21.15.22:0/3819248733 learned_addr learned my addr 172.21.15.22:0/3819248733 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f77fe700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f80cf280 0x7f35f80c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 -- 172.21.15.22:0/3819248733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f80cf280 msgr2=0x7f35f80c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f80cf280 0x7f35f80c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 -- 172.21.15.22:0/3819248733 >> v1:172.21.15.22:6789/0 conn(0x7f35f815c370 legacy=0x7f35f80d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 -- 172.21.15.22:0/3819248733 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80cd500 con 0x7f35f80b10b0 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f67fc700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80b10b0 0x7f35f80ced40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009850 tx=0x7f35e8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbfe429a13beee2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f5ffb700 1 -- 172.21.15.22:0/3819248733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011830 con 0x7f35f80b10b0 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f5ffb700 1 -- 172.21.15.22:0/3819248733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e8011990 con 0x7f35f80b10b0 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f5ffb700 1 -- 172.21.15.22:0/3819248733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011d40 con 0x7f35f80b10b0 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- 172.21.15.22:0/3819248733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80b10b0 msgr2=0x7f35f80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80b10b0 0x7f35f80ced40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009850 tx=0x7f35e8009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- 172.21.15.22:0/3819248733 shutdown_connections 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f80cf280 0x7f35f80c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 --2- 172.21.15.22:0/3819248733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80b10b0 0x7f35f80ced40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- 172.21.15.22:0/3819248733 >> 172.21.15.22:0/3819248733 conn(0x7f35f8069960 msgr2=0x7f35f8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- 172.21.15.22:0/3819248733 shutdown_connections 2023-06-23T18:03:44.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.373+0000 7f35f7fff700 1 -- 172.21.15.22:0/3819248733 wait complete. 2023-06-23T18:03:44.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.377+0000 7f35f7fff700 1 Processor -- start 2023-06-23T18:03:44.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.377+0000 7f35f7fff700 1 -- start start 2023-06-23T18:03:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f80b10b0 0x7f35f80b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f815c370 0x7f35f814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8150d90 con 0x7f35f80cf280 2023-06-23T18:03:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8150f20 con 0x7f35f80b10b0 2023-06-23T18:03:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8156230 con 0x7f35f815c370 2023-06-23T18:03:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40390/0 (socket says 172.21.15.22:40390) 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 -- 172.21.15.22:0/2378032350 learned_addr learned my addr 172.21.15.22:0/2378032350 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f80b10b0 0x7f35f80b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f77fe700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f815c370 0x7f35f814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 -- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f815c370 msgr2=0x7f35f814f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f815c370 0x7f35f814f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 -- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f80b10b0 msgr2=0x7f35f80b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f80b10b0 0x7f35f80b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 -- 172.21.15.22:0/2378032350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80094b0 con 0x7f35f80cf280 2023-06-23T18:03:44.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35e8014040 tx=0x7f35e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35dffff700 1 -- 172.21.15.22:0/2378032350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e80036c0 con 0x7f35f80cf280 2023-06-23T18:03:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35dffff700 1 -- 172.21.15.22:0/2378032350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e801e070 con 0x7f35f80cf280 2023-06-23T18:03:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f8156500 con 0x7f35f80cf280 2023-06-23T18:03:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35dffff700 1 -- 172.21.15.22:0/2378032350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8003a80 con 0x7f35f80cf280 2023-06-23T18:03:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.381+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f8156ac0 con 0x7f35f80cf280 2023-06-23T18:03:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.385+0000 7f35dffff700 1 -- 172.21.15.22:0/2378032350 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f35e8003be0 con 0x7f35f80cf280 2023-06-23T18:03:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.385+0000 7f35dffff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 0x7f35e40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.385+0000 7f35dffff700 1 -- 172.21.15.22:0/2378032350 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f35e8020070 con 0x7f35f80cf280 2023-06-23T18:03:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.385+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 0x7f35e40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 0x7f35e40687a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005930 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 Processor -- start 2023-06-23T18:03:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 -- start start 2023-06-23T18:03:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b8ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7f690 con 0x18f2ae0 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad27c0 con 0x18f2eb0 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1acba20 con 0x18f2710 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40398/0 (socket says 172.21.15.22:40398) 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.389+0000 7f35f67fc700 1 -- 172.21.15.22:0/2695503410 learned_addr learned my addr 172.21.15.22:0/2695503410 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f77fe700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b8ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 -- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 msgr2=0x1b611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 -- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 msgr2=0x1b8ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b8ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 -- 172.21.15.22:0/2695503410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x193e9e0 con 0x18f2ae0 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35e805e0d0 tx=0x7f35e8011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6b3cbba3163cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35dd7fa700 1 -- 172.21.15.22:0/2695503410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e805a270 con 0x18f2ae0 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35dd7fa700 1 -- 172.21.15.22:0/2695503410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e805e600 con 0x18f2ae0 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35dd7fa700 1 -- 172.21.15.22:0/2695503410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e80850f0 con 0x18f2ae0 2023-06-23T18:03:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- 172.21.15.22:0/2695503410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 msgr2=0x1b60c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35e805e0d0 tx=0x7f35e8011e30 comp rx=0 tx=0).stop 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- 172.21.15.22:0/2695503410 shutdown_connections 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b60c70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- 172.21.15.22:0/2695503410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- 172.21.15.22:0/2695503410 >> 172.21.15.22:0/2695503410 conn(0x1b5cab0 msgr2=0x1b670b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- 172.21.15.22:0/2695503410 shutdown_connections 2023-06-23T18:03:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- 172.21.15.22:0/2695503410 wait complete. 2023-06-23T18:03:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- start start 2023-06-23T18:03:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 Processor -- start 2023-06-23T18:03:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b68490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a8df20 con 0x18f2ae0 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a8c890 con 0x18f2eb0 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1977b20 con 0x18f2710 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f3600bde740 10 client.0 _ll_register_callbacks cb 0x1a90350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40404/0 (socket says 172.21.15.22:40404) 2023-06-23T18:03:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f67fc700 1 -- 172.21.15.22:0/122897959 learned_addr learned my addr 172.21.15.22:0/122897959 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b68490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.393+0000 7f35f77fe700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 -- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 msgr2=0x1b68490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b68490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 -- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 msgr2=0x1b656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 -- 172.21.15.22:0/122897959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f8156ac0 con 0x18f2ae0 2023-06-23T18:03:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f67fc700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35e8027040 tx=0x7f35e8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9501dc0109e57b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e805a270 con 0x18f2ae0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f35e8054650 con 0x18f2ae0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ac4a00 con 0x18f2ae0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e8085070 con 0x18f2ae0 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 10 client.4428 fetch_fsmap learned FSMap version 7 2023-06-23T18:03:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x1a91670 con 0x18f2ae0 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ac4a00 con 0x18f2ae0 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f35e807d2a0 con 0x18f2ae0 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 10 client.4428 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 20 client.4428 populate_metadata read hostname 'smithi022' 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x193d790 0x1bb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 4 client.4428 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 --> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ac4720 con 0x193d790 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x193d790 0x1bb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35d67fc700 10 client.4428 ms_handle_connect on v2:172.21.15.22:6835/2486746630 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.397+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x193d790 0x1bb46a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b1019478fb22fb server_cookie=561ce0f1ab78482 in_seq=0 out_seq=0 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f35d67fc700 1 -- 172.21.15.22:0/122897959 <== mds.0 v2:172.21.15.22:6835/2486746630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x1ac4720 con 0x193d790 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f35d67fc700 10 client.4428 handle_command_reply: tid=0 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 client.4428 shutdown 2023-06-23T18:03:44.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 msgr2=0x1b65160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35e8027040 tx=0x7f35e8009eb0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 shutdown_connections 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x193d790 0x1bb46a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1b656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f35d67fc700 0 client.4428 ms_handle_reset on v2:172.21.15.22:6835/2486746630 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2ae0 0x1b65160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 --2- 172.21.15.22:0/122897959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2710 0x1b68490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 >> 172.21.15.22:0/122897959 conn(0x1b5cab0 msgr2=0x1b8e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 shutdown_connections 2023-06-23T18:03:44.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 1 -- 172.21.15.22:0/122897959 wait complete. 2023-06-23T18:03:44.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.401+0000 7f3600bde740 20 client.4428 trim_cache size 0 max 16384 2023-06-23T18:03:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 Processor -- start 2023-06-23T18:03:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 -- start start 2023-06-23T18:03:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bafb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1bbcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a8df20 con 0x18f2710 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ac5d20 con 0x18f2eb0 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1aca8b0 con 0x18f2ae0 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f35f6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.421+0000 7f35f6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40420/0 (socket says 172.21.15.22:40420) 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 -- 172.21.15.22:0/2521253088 learned_addr learned my addr 172.21.15.22:0/2521253088 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f77fe700 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1bbcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f67fc700 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bafb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 -- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 msgr2=0x1bafb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bafb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 -- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 msgr2=0x1bbcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1bbcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 -- 172.21.15.22:0/2521253088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80c6660 con 0x18f2710 2023-06-23T18:03:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x1b8a6f0 tx=0x7f35ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1171fe6ce025a60e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35ddffb700 1 -- 172.21.15.22:0/2521253088 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec028070 con 0x18f2710 2023-06-23T18:03:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35ddffb700 1 -- 172.21.15.22:0/2521253088 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec00fa70 con 0x18f2710 2023-06-23T18:03:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35ddffb700 1 -- 172.21.15.22:0/2521253088 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00fde0 con 0x18f2710 2023-06-23T18:03:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- 172.21.15.22:0/2521253088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 msgr2=0x1baf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x1b8a6f0 tx=0x7f35ec012000 comp rx=0 tx=0).stop 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- 172.21.15.22:0/2521253088 shutdown_connections 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2eb0 0x1bbcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bafb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- 172.21.15.22:0/2521253088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2710 0x1baf610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- 172.21.15.22:0/2521253088 >> 172.21.15.22:0/2521253088 conn(0x1b5cab0 msgr2=0x1b652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- 172.21.15.22:0/2521253088 shutdown_connections 2023-06-23T18:03:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- 172.21.15.22:0/2521253088 wait complete. 2023-06-23T18:03:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- start start 2023-06-23T18:03:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 Processor -- start 2023-06-23T18:03:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2710 0x1bc0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bc1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a9b170 con 0x18f2eb0 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a85b40 con 0x18f2710 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a6a350 con 0x18f2ae0 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f3600bde740 10 client.0 _ll_register_callbacks cb 0x1a79540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:03:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.425+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40430/0 (socket says 172.21.15.22:40430) 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 -- 172.21.15.22:0/676431480 learned_addr learned my addr 172.21.15.22:0/676431480 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2710 0x1bc0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f67fc700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bc1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 -- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 msgr2=0x1bc1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bc1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 -- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2710 msgr2=0x1bc0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2710 0x1bc0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 -- 172.21.15.22:0/676431480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec019040 con 0x18f2eb0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f77fe700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35f0003d40 tx=0x7f35f0003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a86ff6ebfe33c3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f001c070 con 0x18f2eb0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f35f0017a10 con 0x18f2eb0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35e805e600 con 0x18f2eb0 2023-06-23T18:03:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f000e040 con 0x18f2eb0 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 10 client.4434 fetch_fsmap learned FSMap version 7 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x1b67f00 con 0x18f2eb0 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e805e600 con 0x18f2eb0 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f35f0017bd0 con 0x18f2eb0 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 10 client.4434 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 20 client.4434 populate_metadata read hostname 'smithi022' 2023-06-23T18:03:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1bbeff0 0x1c086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 4 client.4434 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 --> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] -- command(tid 0: {"prefix": "status"}) v1 -- 0x194a4e0 con 0x1bbeff0 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1bbeff0 0x1c086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35f6ffd700 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1bbeff0 0x1c086c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81ae3565a1e7913 server_cookie=f7cd44f7aba4a802 in_seq=0 out_seq=0 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 10 client.4434 ms_handle_connect on v2:172.21.15.22:6835/2486746630 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 1 -- 172.21.15.22:0/676431480 <== mds.0 v2:172.21.15.22:6835/2486746630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x194a4e0 con 0x1bbeff0 2023-06-23T18:03:44.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 10 client.4434 handle_command_reply: tid=0 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 client.4434 shutdown 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 msgr2=0x1bc2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35f0003d40 tx=0x7f35f0003d70 comp rx=0 tx=0).stop 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 shutdown_connections 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1bbeff0 0x1c086c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x18f2eb0 0x1bc2470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f35d67fc700 0 client.4434 ms_handle_reset on v2:172.21.15.22:6835/2486746630 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x18f2ae0 0x1bc1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 --2- 172.21.15.22:0/676431480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x18f2710 0x1bc0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.429+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 >> 172.21.15.22:0/676431480 conn(0x1b5cab0 msgr2=0x1b652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 shutdown_connections 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f3600bde740 1 -- 172.21.15.22:0/676431480 wait complete. 2023-06-23T18:03:44.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f3600bde740 20 client.4434 trim_cache size 0 max 16384 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "e4fadccd-0593-4464-bce3-4c0f7c738354", 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4263, 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-06-23T18:03:44.451 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 6.7030419569999999, 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 7, 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 24, 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 23, 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 11.080108107999999 2023-06-23T18:03:44.452 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:03:44.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 msgr2=0x7f35e40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 0x7f35e40687a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005930 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2023-06-23T18:03:44.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 msgr2=0x7f35f8155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35e8014040 tx=0x7f35e8004770 comp rx=0 tx=0).stop 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 shutdown_connections 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35f815c370 0x7f35f814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f80cf280 0x7f35f8155cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35e40662f0 0x7f35e40687a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 --2- 172.21.15.22:0/2378032350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f80b10b0 0x7f35f80b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 >> 172.21.15.22:0/2378032350 conn(0x7f35f8069960 msgr2=0x7f35f80c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 shutdown_connections 2023-06-23T18:03:44.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.433+0000 7f35f7fff700 1 -- 172.21.15.22:0/2378032350 wait complete. 2023-06-23T18:03:44.463 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:03:44.464 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:44.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:03:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 Processor -- start 2023-06-23T18:03:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 -- start start 2023-06-23T18:03:44.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f96a8157f00 0x7f96a80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 0x7f96a80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a80cdb20 con 0x7f96a8157f00 2023-06-23T18:03:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a80c9780 con 0x7f96a80cd160 2023-06-23T18:03:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96af435700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80c4c50 con 0x7f96a815baf0 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96ae433700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f96a8157f00 0x7f96a80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96ae433700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f96a8157f00 0x7f96a80cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40440/0 (socket says 172.21.15.22:40440) 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.573+0000 7f96ae433700 1 -- 172.21.15.22:0/2672921172 learned_addr learned my addr 172.21.15.22:0/2672921172 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96adc32700 1 --2- 172.21.15.22:0/2672921172 >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 0x7f96a80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 -- 172.21.15.22:0/2672921172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 msgr2=0x7f96a815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96aec34700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 -- 172.21.15.22:0/2672921172 >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 msgr2=0x7f96a80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 --2- 172.21.15.22:0/2672921172 >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 0x7f96a80cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 -- 172.21.15.22:0/2672921172 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80c4f30 con 0x7f96a8157f00 2023-06-23T18:03:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96adc32700 1 --2- 172.21.15.22:0/2672921172 >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 0x7f96a80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96aec34700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ae433700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a8157f00 0x7f96a80cc360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96a806dd90 tx=0x7f96a4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a16cc8bb566012 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ad431700 1 -- 172.21.15.22:0/2672921172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400b8a0 con 0x7f96a8157f00 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ad431700 1 -- 172.21.15.22:0/2672921172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a400ba00 con 0x7f96a8157f00 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96ad431700 1 -- 172.21.15.22:0/2672921172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400bd90 con 0x7f96a8157f00 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- 172.21.15.22:0/2672921172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a8157f00 msgr2=0x7f96a80cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a8157f00 0x7f96a80cc360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96a806dd90 tx=0x7f96a4009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- 172.21.15.22:0/2672921172 shutdown_connections 2023-06-23T18:03:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 --2- 172.21.15.22:0/2672921172 >> v2:172.21.15.169:3300/0 conn(0x7f96a80cd160 0x7f96a80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 --2- 172.21.15.22:0/2672921172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a8157f00 0x7f96a80cc360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- 172.21.15.22:0/2672921172 >> 172.21.15.22:0/2672921172 conn(0x7f96a80cb2a0 msgr2=0x7f96a80cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- 172.21.15.22:0/2672921172 shutdown_connections 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- 172.21.15.22:0/2672921172 wait complete. 2023-06-23T18:03:44.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 Processor -- start 2023-06-23T18:03:44.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.577+0000 7f96af435700 1 -- start start 2023-06-23T18:03:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 0x7f96a80c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a8157f00 0x7f96a8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a815b020 con 0x7f96a815baf0 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80c7930 con 0x7f96a80cd160 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8159f80 con 0x7f96a8157f00 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40446/0 (socket says 172.21.15.22:40446) 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96adc32700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a8157f00 0x7f96a8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96ae433700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 0x7f96a80c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 -- 172.21.15.22:0/3353087688 learned_addr learned my addr 172.21.15.22:0/3353087688 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 -- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a8157f00 msgr2=0x7f96a8151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a8157f00 0x7f96a8151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 -- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 msgr2=0x7f96a80c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 0x7f96a80c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 -- 172.21.15.22:0/3353087688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4009570 con 0x7f96a815baf0 2023-06-23T18:03:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96ae433700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 0x7f96a80c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96aec34700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96a000cd50 tx=0x7f96a000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a000cea0 con 0x7f96a815baf0 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000f480 con 0x7f96a815baf0 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a815a260 con 0x7f96a815baf0 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a000f830 con 0x7f96a815baf0 2023-06-23T18:03:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.581+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a815a820 con 0x7f96a815baf0 2023-06-23T18:03:44.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a804e4c0 con 0x7f96a815baf0 2023-06-23T18:03:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f96a000b040 con 0x7f96a815baf0 2023-06-23T18:03:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f969effd700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 0x7f968c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96a0084720 con 0x7f96a815baf0 2023-06-23T18:03:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f96ae433700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 0x7f968c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.585+0000 7f96ae433700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 0x7f968c0687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000c00 tx=0x7f96a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:44.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.593+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f96a0051820 con 0x7f96a815baf0 2023-06-23T18:03:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.801+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96a80c3c20 con 0x7f96a815baf0 2023-06-23T18:03:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.801+0000 7f969effd700 1 -- 172.21.15.22:0/3353087688 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f96a001d070 con 0x7f96a815baf0 2023-06-23T18:03:44.815 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-23T18:03:44.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 msgr2=0x7f968c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 0x7f968c0687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000c00 tx=0x7f96a4008030 comp rx=0 tx=0).stop 2023-06-23T18:03:44.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 msgr2=0x7f96a815bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96a000cd50 tx=0x7f96a000a560 comp rx=0 tx=0).stop 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 shutdown_connections 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a815baf0 0x7f96a815bf70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.805+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a8157f00 0x7f96a8151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.809+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f968c0662f0 0x7f968c0687a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.809+0000 7f96af435700 1 --2- 172.21.15.22:0/3353087688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a80cd160 0x7f96a80c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.809+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 >> 172.21.15.22:0/3353087688 conn(0x7f96a80cb2a0 msgr2=0x7f96a80b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.809+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 shutdown_connections 2023-06-23T18:03:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:44.809+0000 7f96af435700 1 -- 172.21.15.22:0/3353087688 wait complete. 2023-06-23T18:03:44.836 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-06-23T18:03:44.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:03:44.837 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-06-23T18:03:44.837 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-06-23T18:03:44.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:44.837 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-06-23T18:03:44.838 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:03:44.838 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:03:44.838 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-06-23T18:03:44.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:44.839 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-06-23T18:03:44.844 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:03:44.844 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:03:44.844 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:03:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42580sec preferred_lft 42580sec 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:03:44.846 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:03:44.847 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:03:44.847 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:03:44.847 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:c1:01:7a:e8 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:03:44.847 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:03:44.847 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:03:44.847 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:03:44.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:44.848 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:03:44.848 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:03:44.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:03:44.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-06-23T18:03:44.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-06-23T18:03:44.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:03:44.850 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:03:44.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:44.894 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:03:44.950 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-06-23T18:03:44.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:44.951 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-06-23T18:03:44.984 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-06-23T18:03:44.984 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:03:44.985 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-06-23T18:03:44.985 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:03:44.985 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-06-23T18:03:44.985 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:03:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:44.986 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:03:44.986 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:03:44.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-23T18:03:44.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-23T18:03:44.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-23T18:03:44.987 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:03:45.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.027 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-06-23T18:03:45.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.080 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-06-23T18:03:45.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.088 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:03:45.088 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:03:45.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-06-23T18:03:45.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-06-23T18:03:45.089 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:03:45.118 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-06-23T18:03:45.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.119 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:03:45.119 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:03:45.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-06-23T18:03:45.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:03:45.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-06-23T18:03:45.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-06-23T18:03:45.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-06-23T18:03:45.121 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:03:45.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.233 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:03:45.233 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:03:45.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-06-23T18:03:45.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:03:45.234 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:03:45.263 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-06-23T18:03:45.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:03:45.264 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:45.269 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-06-23T18:03:45.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:03:45.270 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:45.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:03:45.333 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:03:45.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:03:45.350 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:03:45.351 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:03:45.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.352 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:03:45.357 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:03:45.358 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-06-23T18:03:45.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:03:45.418 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:03:45.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:03:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:03:45.433 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:03:45.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:45.435 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:03:45.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:2023-06-23T18:03:45.477+0000 7fdba47fb640 -1 init, newargv = 0x55e2bc368650 newargc=15 2023-06-23T18:03:45.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[41389]: starting ceph client 2023-06-23T18:03:45.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[41389]: starting fuse 2023-06-23T18:03:46.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:03:46.562 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:03:46.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:03:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:03:46.580 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:03:46.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:46.581 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:03:46.588 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:03:46.589 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-06-23T18:03:46.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:03:46.590 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-06-23T18:03:46.591 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-06-23T18:03:46.592 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-06-23T18:03:46.592 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-06-23T18:03:46.592 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:03:46.593 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-06-23T18:03:46.594 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-06-23T18:03:46.594 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-06-23T18:03:46.595 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-06-23T18:03:46.595 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:03:46.595 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-06-23T18:03:46.596 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-06-23T18:03:46.596 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-06-23T18:03:46.596 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-06-23T18:03:46.597 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:03:46.597 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-06-23T18:03:46.598 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:03:46.598 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:03:46.598 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:03:46.599 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-06-23T18:03:46.599 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-06-23T18:03:46.600 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-06-23T18:03:46.600 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:03:46.601 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:03:46.601 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.admin")) 2023-06-23T18:03:46.601 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-06-23T18:03:46.687 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.admin.41389.asok 2023-06-23T18:03:46.873 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41389.asok 2023-06-23T18:03:46.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:46.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41389.asok status 2023-06-23T18:03:47.023 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:03:47.024 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-06-23T18:03:47.024 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:03:47.025 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:03:47.025 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "admin", 2023-06-23T18:03:47.025 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-06-23T18:03:47.025 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "41389", 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-06-23T18:03:47.026 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4446, 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4446 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:03:47.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:03:47.028 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:03:47.028 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 489048646 2023-06-23T18:03:47.028 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:03:47.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:03:47.029 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:03:47.029 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:03:47.029 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:03:47.029 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 489048646 2023-06-23T18:03:47.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4446 192.168.0.1:0/489048646", 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/489048646", 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 7, 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 24, 2023-06-23T18:03:47.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:03:47.031 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-06-23T18:03:47.031 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-06-23T18:03:47.031 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:03:47.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:47.033 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-06-23T18:03:47.034 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:47.041 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-06-23T18:03:47.042 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:47.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:47.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:47.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:47.056 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-06-23T18:03:47.138 INFO:teuthology.orchestra.run.smithi022.stdout:# file: . 2023-06-23T18:03:47.138 INFO:teuthology.orchestra.run.smithi022.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-06-23T18:03:47.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:47.139 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:47.139 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:03:47.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:47.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 Processor -- start 2023-06-23T18:03:47.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- start start 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fae1c0ceb60 0x7fae1c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0ceff0 con 0x7fae1c0c9d30 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c15d990 con 0x7fae1c0cb620 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0cb9f0 con 0x7fae1c0ceb60 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fae1c0ceb60 0x7fae1c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fae1c0ceb60 0x7fae1c15de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47462/0 (socket says 172.21.15.22:47462) 2023-06-23T18:03:47.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 -- 172.21.15.22:0/1369251477 learned_addr learned my addr 172.21.15.22:0/1369251477 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 -- 172.21.15.22:0/1369251477 >> v1:172.21.15.169:6789/0 conn(0x7fae1c0cb620 legacy=0x7fae1c0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 -- 172.21.15.22:0/1369251477 >> v1:172.21.15.22:6789/0 conn(0x7fae1c0c9d30 legacy=0x7fae1c070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 -- 172.21.15.22:0/1369251477 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c06c670 con 0x7fae1c0ceb60 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1259c700 1 --2- 172.21.15.22:0/1369251477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0ceb60 0x7fae1c15de20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae04009910 tx=0x7fae04009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537907710fcc55e2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1a59c700 1 -- 172.21.15.22:0/1369251477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0400ba60 con 0x7fae1c0ceb60 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1a59c700 1 -- 172.21.15.22:0/1369251477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400bbc0 con 0x7fae1c0ceb60 2023-06-23T18:03:47.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae1a59c700 1 -- 172.21.15.22:0/1369251477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04009e60 con 0x7fae1c0ceb60 2023-06-23T18:03:47.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- 172.21.15.22:0/1369251477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0ceb60 msgr2=0x7fae1c15de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 --2- 172.21.15.22:0/1369251477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0ceb60 0x7fae1c15de20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae04009910 tx=0x7fae04009c20 comp rx=0 tx=0).stop 2023-06-23T18:03:47.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- 172.21.15.22:0/1369251477 shutdown_connections 2023-06-23T18:03:47.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 --2- 172.21.15.22:0/1369251477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0ceb60 0x7fae1c15de20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- 172.21.15.22:0/1369251477 >> 172.21.15.22:0/1369251477 conn(0x7fae1c069960 msgr2=0x7fae1c069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:47.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- 172.21.15.22:0/1369251477 shutdown_connections 2023-06-23T18:03:47.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.261+0000 7fae2107d700 1 -- 172.21.15.22:0/1369251477 wait complete. 2023-06-23T18:03:47.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.265+0000 7fae2107d700 1 Processor -- start 2023-06-23T18:03:47.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.265+0000 7fae2107d700 1 -- start start 2023-06-23T18:03:47.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae1c0c9d30 0x7fae1c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 0x7fae1c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c15b4f0 con 0x7fae1c0cb620 2023-06-23T18:03:47.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c15b7d0 con 0x7fae1c0c9d30 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c15bab0 con 0x7fae1c0ceb60 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47474/0 (socket says 172.21.15.22:47474) 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 -- 172.21.15.22:0/1956310791 learned_addr learned my addr 172.21.15.22:0/1956310791 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1ad9d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae1c0c9d30 0x7fae1c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1b59e700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 0x7fae1c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 -- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 msgr2=0x7fae1c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 0x7fae1c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 -- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae1c0c9d30 msgr2=0x7fae1c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae1c0c9d30 0x7fae1c15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 -- 172.21.15.22:0/1956310791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae04009570 con 0x7fae1c0cb620 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1b59e700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 0x7fae1c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae1259c700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae04000c00 tx=0x7fae04004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040036c0 con 0x7fae1c0cb620 2023-06-23T18:03:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0401f070 con 0x7fae1c0cb620 2023-06-23T18:03:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c15bd90 con 0x7fae1c0cb620 2023-06-23T18:03:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04003a80 con 0x7fae1c0cb620 2023-06-23T18:03:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.269+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c14f9f0 con 0x7fae1c0cb620 2023-06-23T18:03:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c15c2b0 con 0x7fae1c0cb620 2023-06-23T18:03:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fae04003be0 con 0x7fae1c0cb620 2023-06-23T18:03:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae13fff700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 0x7fadfc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae1ad9d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 0x7fadfc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fae04021070 con 0x7fae1c0cb620 2023-06-23T18:03:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.273+0000 7fae1ad9d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 0x7fadfc0687a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae0c000f60 tx=0x7fae0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.277+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fae1c15c2b0 con 0x7fae1c0cb620 2023-06-23T18:03:47.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.445+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae1c15c2b0 con 0x7fae1c0cb620 2023-06-23T18:03:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.445+0000 7fae13fff700 1 -- 172.21.15.22:0/1956310791 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fae04073ea0 con 0x7fae1c0cb620 2023-06-23T18:03:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:47.460 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.449+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 msgr2=0x7fadfc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.449+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 0x7fadfc0687a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae0c000f60 tx=0x7fae0c009040 comp rx=0 tx=0).stop 2023-06-23T18:03:47.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.449+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 msgr2=0x7fae1c14f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.449+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae04000c00 tx=0x7fae04004770 comp rx=0 tx=0).stop 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 shutdown_connections 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae1c0ceb60 0x7fae1c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae1c0cb620 0x7fae1c14f4b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fadfc0662f0 0x7fadfc0687a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 --2- 172.21.15.22:0/1956310791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae1c0c9d30 0x7fae1c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 >> 172.21.15.22:0/1956310791 conn(0x7fae1c069960 msgr2=0x7fae1c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 shutdown_connections 2023-06-23T18:03:47.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.453+0000 7fae2107d700 1 -- 172.21.15.22:0/1956310791 wait complete. 2023-06-23T18:03:47.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:03:47.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:03:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 Processor -- start 2023-06-23T18:03:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 -- start start 2023-06-23T18:03:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa9b40b1920 0x7fa9b40cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b40d1d40 con 0x7fa9b40d0a20 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40b1cf0 con 0x7fa9b40b1920 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40d31e0 con 0x7fa9b40cca20 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b37fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa9b40d0a20 0x7fa9b415f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57098/0 (socket says 172.21.15.22:57098) 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b37fe700 1 -- 172.21.15.22:0/313077638 learned_addr learned my addr 172.21.15.22:0/313077638 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/313077638 >> v2:172.21.15.22:3300/0 conn(0x7fa9b40b1920 0x7fa9b40cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b27fc700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b1ffb700 1 -- 172.21.15.22:0/313077638 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424867772 0 0) 0x7fa9b40d1d40 con 0x7fa9b40d0a20 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b1ffb700 1 -- 172.21.15.22:0/313077638 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa99c0035d0 con 0x7fa9b40d0a20 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/313077638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 msgr2=0x7fa9b40d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/313077638 >> v1:172.21.15.22:6789/0 conn(0x7fa9b40d0a20 legacy=0x7fa9b415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/313077638 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b40cf130 con 0x7fa9b40b1920 2023-06-23T18:03:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b40cffa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009850 tx=0x7fa9a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d42118ac41392d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b1ffb700 1 -- 172.21.15.22:0/313077638 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011830 con 0x7fa9b40b1920 2023-06-23T18:03:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b1ffb700 1 -- 172.21.15.22:0/313077638 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4011990 con 0x7fa9b40b1920 2023-06-23T18:03:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.593+0000 7fa9b1ffb700 1 -- 172.21.15.22:0/313077638 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011d40 con 0x7fa9b40b1920 2023-06-23T18:03:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- 172.21.15.22:0/313077638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 msgr2=0x7fa9b40cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b40cffa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009850 tx=0x7fa9a4009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- 172.21.15.22:0/313077638 shutdown_connections 2023-06-23T18:03:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/313077638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b40cffa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- 172.21.15.22:0/313077638 >> 172.21.15.22:0/313077638 conn(0x7fa9b4069960 msgr2=0x7fa9b4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- 172.21.15.22:0/313077638 shutdown_connections 2023-06-23T18:03:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- 172.21.15.22:0/313077638 wait complete. 2023-06-23T18:03:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 Processor -- start 2023-06-23T18:03:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.597+0000 7fa9b3fff700 1 -- start start 2023-06-23T18:03:47.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9b40d0a20 0x7fa9b40b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b415f1d0 con 0x7fa9b40b1920 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b414e300 con 0x7fa9b40d0a20 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b414ed70 con 0x7fa9b40cca20 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47490/0 (socket says 172.21.15.22:47490) 2023-06-23T18:03:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/2194383262 learned_addr learned my addr 172.21.15.22:0/2194383262 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b27fc700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b37fe700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9b40d0a20 0x7fa9b40b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 msgr2=0x7fa9b40b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9b40d0a20 msgr2=0x7fa9b40b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9b40d0a20 0x7fa9b40b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 -- 172.21.15.22:0/2194383262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a40094b0 con 0x7fa9b40b1920 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b27fc700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b2ffd700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406c750 tx=0x7fa9a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4004300 con 0x7fa9b40b1920 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a401f070 con 0x7fa9b40b1920 2023-06-23T18:03:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b415e8a0 con 0x7fa9b40b1920 2023-06-23T18:03:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40036c0 con 0x7fa9b40b1920 2023-06-23T18:03:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.601+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b415ee60 con 0x7fa9b40b1920 2023-06-23T18:03:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b404e4c0 con 0x7fa9b40b1920 2023-06-23T18:03:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa9a4021030 con 0x7fa9b40b1920 2023-06-23T18:03:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa99bfff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 0x7fa9a00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa9a4085410 con 0x7fa9b40b1920 2023-06-23T18:03:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa9b27fc700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 0x7fa9a00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.605+0000 7fa9b27fc700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 0x7fa9a00687d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40b4020 tx=0x7fa9a8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.613+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa9a4052750 con 0x7fa9b40b1920 2023-06-23T18:03:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.773+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9b40b46d0 con 0x7fa9b40b1920 2023-06-23T18:03:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.773+0000 7fa99bfff700 1 -- 172.21.15.22:0/2194383262 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9a4006ce0 con 0x7fa9b40b1920 2023-06-23T18:03:47.786 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:03:47.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 msgr2=0x7fa9a00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 0x7fa9a00687d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40b4020 tx=0x7fa9a8009240 comp rx=0 tx=0).stop 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 msgr2=0x7fa9b415c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406c750 tx=0x7fa9a4004770 comp rx=0 tx=0).stop 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 shutdown_connections 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9a0066320 0x7fa9a00687d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9b40d0a20 0x7fa9b40b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9b40cca20 0x7fa9b40b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 --2- 172.21.15.22:0/2194383262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9b40b1920 0x7fa9b415c0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 >> 172.21.15.22:0/2194383262 conn(0x7fa9b4069960 msgr2=0x7fa9b4160000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:47.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 shutdown_connections 2023-06-23T18:03:47.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.777+0000 7fa9b3fff700 1 -- 172.21.15.22:0/2194383262 wait complete. 2023-06-23T18:03:47.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:03:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 Processor -- start 2023-06-23T18:03:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 -- start start 2023-06-23T18:03:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 0x7f8adc0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0b1a50 con 0x7f8adc160890 2023-06-23T18:03:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc161830 con 0x7f8adc0b1140 2023-06-23T18:03:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ae1ccd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc161550 con 0x7f8adc0706c0 2023-06-23T18:03:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adb7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 0x7f8adc0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adaffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8adc160890 0x7f8adc160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:56440/0 (socket says 172.21.15.22:56440) 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adaffd700 1 -- 172.21.15.22:0/2860924315 learned_addr learned my addr 172.21.15.22:0/2860924315 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:34952/0 (socket says 172.21.15.22:34952) 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adb7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 0x7f8adc0b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:40822/0 (socket says 172.21.15.22:40822) 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 -- 172.21.15.22:0/2860924315 >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 msgr2=0x7f8adc0b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 --2- 172.21.15.22:0/2860924315 >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 0x7f8adc0b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 -- 172.21.15.22:0/2860924315 >> v1:172.21.15.169:6789/0 conn(0x7f8adc160890 legacy=0x7f8adc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 -- 172.21.15.22:0/2860924315 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc06fbe0 con 0x7f8adc0706c0 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8adbfff700 1 --2- 172.21.15.22:0/2860924315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009d90 tx=0x7f8ad400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14ab2dd7921cf93 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.908+0000 7f8ada7fc700 1 -- 172.21.15.22:0/2860924315 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400e070 con 0x7f8adc0706c0 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ada7fc700 1 -- 172.21.15.22:0/2860924315 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400bb20 con 0x7f8adc0706c0 2023-06-23T18:03:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ada7fc700 1 -- 172.21.15.22:0/2860924315 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4012430 con 0x7f8adc0706c0 2023-06-23T18:03:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2860924315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 msgr2=0x7f8adc070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2860924315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009d90 tx=0x7f8ad400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2860924315 shutdown_connections 2023-06-23T18:03:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2860924315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc0706c0 0x7f8adc070b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2860924315 >> v2:172.21.15.169:3300/0 conn(0x7f8adc0b1140 0x7f8adc0b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2860924315 >> 172.21.15.22:0/2860924315 conn(0x7f8adc060560 msgr2=0x7f8adc06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2860924315 shutdown_connections 2023-06-23T18:03:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2860924315 wait complete. 2023-06-23T18:03:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 Processor -- start 2023-06-23T18:03:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- start start 2023-06-23T18:03:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8adc0b1140 0x7f8adc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc160890 0x7f8adc070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0cdf50 con 0x7f8adc0706c0 2023-06-23T18:03:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1f68b0 con 0x7f8adc0b1140 2023-06-23T18:03:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8ae1ccd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0cd760 con 0x7f8adc160890 2023-06-23T18:03:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.912+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47502/0 (socket says 172.21.15.22:47502) 2023-06-23T18:03:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 -- 172.21.15.22:0/2647214307 learned_addr learned my addr 172.21.15.22:0/2647214307 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adaffd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8adc0b1140 0x7f8adc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adbfff700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc160890 0x7f8adc070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 -- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc160890 msgr2=0x7f8adc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc160890 0x7f8adc070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 -- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8adc0b1140 msgr2=0x7f8adc0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8adc0b1140 0x7f8adc0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 -- 172.21.15.22:0/2647214307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009a70 con 0x7f8adc0706c0 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8adb7fe700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e840 tx=0x7f8ad000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00095a0 con 0x7f8adc0706c0 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc1f6270 con 0x7f8adc0706c0 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0013680 con 0x7f8adc0706c0 2023-06-23T18:03:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc1f64d0 con 0x7f8adc0706c0 2023-06-23T18:03:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00139f0 con 0x7f8adc0706c0 2023-06-23T18:03:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.916+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc1f6b20 con 0x7f8adc0706c0 2023-06-23T18:03:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.920+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8ad0013b50 con 0x7f8adc0706c0 2023-06-23T18:03:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.920+0000 7f8ac3fff700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 0x7f8ac80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.920+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8ad0083ef0 con 0x7f8adc0706c0 2023-06-23T18:03:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.920+0000 7f8adaffd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 0x7f8ac80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.920+0000 7f8adaffd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 0x7f8ac80687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8adc1f9790 tx=0x7f8acc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:47.928+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8ad0054a80 con 0x7f8adc0706c0 2023-06-23T18:03:48.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.092+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8adc04dc70 con 0x7f8adc0706c0 2023-06-23T18:03:48.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.092+0000 7f8ac3fff700 1 -- 172.21.15.22:0/2647214307 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ad0054610 con 0x7f8adc0706c0 2023-06-23T18:03:48.108 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:03:48.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 msgr2=0x7f8ac80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 0x7f8ac80687a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8adc1f9790 tx=0x7f8acc009040 comp rx=0 tx=0).stop 2023-06-23T18:03:48.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 msgr2=0x7f8adc0cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e840 tx=0x7f8ad000eb50 comp rx=0 tx=0).stop 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 shutdown_connections 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8adc160890 0x7f8adc070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8ac80662f0 0x7f8ac80687a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8adc0b1140 0x7f8adc0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 --2- 172.21.15.22:0/2647214307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8adc0706c0 0x7f8adc0cae00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 >> 172.21.15.22:0/2647214307 conn(0x7f8adc060560 msgr2=0x7f8adc0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 shutdown_connections 2023-06-23T18:03:48.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.096+0000 7f8ae1ccd700 1 -- 172.21.15.22:0/2647214307 wait complete. 2023-06-23T18:03:48.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:03:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 Processor -- start 2023-06-23T18:03:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 -- start start 2023-06-23T18:03:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 0x7fa18c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa18c0cc7b0 0x7fa18c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0cdba0 con 0x7fa18c160430 2023-06-23T18:03:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0cf520 con 0x7fa18c0cc7b0 2023-06-23T18:03:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa192a9b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0cf800 con 0x7fa18c0b2470 2023-06-23T18:03:48.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa191a99700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa18c160430 0x7fa18c0b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57112/0 (socket says 172.21.15.22:57112) 2023-06-23T18:03:48.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa191a99700 1 -- 172.21.15.22:0/3548975706 learned_addr learned my addr 172.21.15.22:0/3548975706 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa191298700 1 --2- 172.21.15.22:0/3548975706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 0x7fa18c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.228+0000 7fa19229a700 1 --2- 172.21.15.22:0/3548975706 >> v2:172.21.15.169:3300/0 conn(0x7fa18c0cc7b0 0x7fa18c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa190a97700 1 -- 172.21.15.22:0/3548975706 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955979164 0 0) 0x7fa18c0cdba0 con 0x7fa18c160430 2023-06-23T18:03:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa190a97700 1 -- 172.21.15.22:0/3548975706 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1780035d0 con 0x7fa18c160430 2023-06-23T18:03:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa191298700 1 -- 172.21.15.22:0/3548975706 >> v2:172.21.15.169:3300/0 conn(0x7fa18c0cc7b0 msgr2=0x7fa18c06f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa191298700 1 --2- 172.21.15.22:0/3548975706 >> v2:172.21.15.169:3300/0 conn(0x7fa18c0cc7b0 0x7fa18c06f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa191298700 1 -- 172.21.15.22:0/3548975706 >> v1:172.21.15.22:6789/0 conn(0x7fa18c160430 legacy=0x7fa18c0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa191298700 1 -- 172.21.15.22:0/3548975706 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0cfae0 con 0x7fa18c0b2470 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa191298700 1 --2- 172.21.15.22:0/3548975706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 0x7fa18c06f3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa174009850 tx=0x7fa174009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91be757e9730131b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa190a97700 1 -- 172.21.15.22:0/3548975706 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011830 con 0x7fa18c0b2470 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa190a97700 1 -- 172.21.15.22:0/3548975706 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174011990 con 0x7fa18c0b2470 2023-06-23T18:03:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa190a97700 1 -- 172.21.15.22:0/3548975706 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011d40 con 0x7fa18c0b2470 2023-06-23T18:03:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 -- 172.21.15.22:0/3548975706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 msgr2=0x7fa18c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 --2- 172.21.15.22:0/3548975706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 0x7fa18c06f3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa174009850 tx=0x7fa174009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 -- 172.21.15.22:0/3548975706 shutdown_connections 2023-06-23T18:03:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 --2- 172.21.15.22:0/3548975706 >> v2:172.21.15.169:3300/0 conn(0x7fa18c0cc7b0 0x7fa18c06f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 --2- 172.21.15.22:0/3548975706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c0b2470 0x7fa18c06f3e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 -- 172.21.15.22:0/3548975706 >> 172.21.15.22:0/3548975706 conn(0x7fa18c0ccd60 msgr2=0x7fa18c0cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 -- 172.21.15.22:0/3548975706 shutdown_connections 2023-06-23T18:03:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.232+0000 7fa192a9b700 1 -- 172.21.15.22:0/3548975706 wait complete. 2023-06-23T18:03:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 Processor -- start 2023-06-23T18:03:48.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- start start 2023-06-23T18:03:48.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 0x7fa18c163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c160430 0x7fa18c160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0cbea0 con 0x7fa18c0b2470 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c160d40 con 0x7fa18c0cc7b0 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0b4820 con 0x7fa18c160430 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47508/0 (socket says 172.21.15.22:47508) 2023-06-23T18:03:48.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 -- 172.21.15.22:0/988692206 learned_addr learned my addr 172.21.15.22:0/988692206 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa19229a700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c160430 0x7fa18c160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191298700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 0x7fa18c163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 -- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c160430 msgr2=0x7fa18c160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c160430 0x7fa18c160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 -- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 msgr2=0x7fa18c163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 0x7fa18c163320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 -- 172.21.15.22:0/988692206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740094b0 con 0x7fa18c0b2470 2023-06-23T18:03:48.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191298700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 0x7fa18c163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:48.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa191a99700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00cbc0 tx=0x7fa17c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00a7b0 con 0x7fa18c0b2470 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c00a910 con 0x7fa18c0b2470 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c0b4a70 con 0x7fa18c0b2470 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00aca0 con 0x7fa18c0b2470 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.236+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c0b5030 con 0x7fa18c0b2470 2023-06-23T18:03:48.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c04e4c0 con 0x7fa18c0b2470 2023-06-23T18:03:48.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa17c01b3b0 con 0x7fa18c0b2470 2023-06-23T18:03:48.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa18a7fc700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 0x7fa16c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa17c0839c0 con 0x7fa18c0b2470 2023-06-23T18:03:48.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa191298700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 0x7fa16c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.240+0000 7fa191298700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 0x7fa16c0687d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b2860 tx=0x7fa174008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.248+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa17c050b40 con 0x7fa18c0b2470 2023-06-23T18:03:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.408+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa18c06c5d0 con 0x7fa18c0b2470 2023-06-23T18:03:48.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa18a7fc700 1 -- 172.21.15.22:0/988692206 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa17c054160 con 0x7fa18c0b2470 2023-06-23T18:03:48.424 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 msgr2=0x7fa16c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 0x7fa16c0687d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b2860 tx=0x7fa174008030 comp rx=0 tx=0).stop 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 msgr2=0x7fa18c15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00cbc0 tx=0x7fa17c00cbf0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 shutdown_connections 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18c160430 0x7fa18c160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa16c066320 0x7fa16c0687d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa18c0cc7b0 0x7fa18c163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 --2- 172.21.15.22:0/988692206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa18c0b2470 0x7fa18c15cde0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 >> 172.21.15.22:0/988692206 conn(0x7fa18c0ccd60 msgr2=0x7fa18c0cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 shutdown_connections 2023-06-23T18:03:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.412+0000 7fa192a9b700 1 -- 172.21.15.22:0/988692206 wait complete. 2023-06-23T18:03:48.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:03:48.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 Processor -- start 2023-06-23T18:03:48.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 -- start start 2023-06-23T18:03:48.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d0c158b20 0x7f1d0c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158ef0 0x7f1d0c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d0c070740 0x7f1d0c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c0cdf20 con 0x7f1d0c070740 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c071100 con 0x7f1d0c158b20 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d11189700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c06eab0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d0c070740 0x7f1d0c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d0c070740 0x7f1d0c070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47518/0 (socket says 172.21.15.22:47518) 2023-06-23T18:03:48.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 -- 172.21.15.22:0/1038447068 learned_addr learned my addr 172.21.15.22:0/1038447068 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d0bfff700 1 --2- 172.21.15.22:0/1038447068 >> v2:172.21.15.169:3300/0 conn(0x7f1d0c158b20 0x7f1d0c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d037fe700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158ef0 0x7f1d0c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 -- 172.21.15.22:0/1038447068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158ef0 msgr2=0x7f1d0c0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158ef0 0x7f1d0c0b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 -- 172.21.15.22:0/1038447068 >> v2:172.21.15.169:3300/0 conn(0x7f1d0c158b20 msgr2=0x7f1d0c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 --2- 172.21.15.22:0/1038447068 >> v2:172.21.15.169:3300/0 conn(0x7f1d0c158b20 0x7f1d0c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 -- 172.21.15.22:0/1038447068 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0cd680 con 0x7f1d0c070740 2023-06-23T18:03:48.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.540+0000 7f1d10988700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c070740 0x7f1d0c070bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0701f0 tx=0x7f1d0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932d692e47f59574 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1038447068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400e070 con 0x7f1d0c070740 2023-06-23T18:03:48.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1038447068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0400b910 con 0x7f1d0c070740 2023-06-23T18:03:48.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1038447068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400bcc0 con 0x7f1d0c070740 2023-06-23T18:03:48.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- 172.21.15.22:0/1038447068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c070740 msgr2=0x7f1d0c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c070740 0x7f1d0c070bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0701f0 tx=0x7f1d0400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:48.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- 172.21.15.22:0/1038447068 shutdown_connections 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c070740 0x7f1d0c070bc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 --2- 172.21.15.22:0/1038447068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158ef0 0x7f1d0c0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 --2- 172.21.15.22:0/1038447068 >> v2:172.21.15.169:3300/0 conn(0x7f1d0c158b20 0x7f1d0c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- 172.21.15.22:0/1038447068 >> 172.21.15.22:0/1038447068 conn(0x7f1d0c06b700 msgr2=0x7f1d0c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- 172.21.15.22:0/1038447068 shutdown_connections 2023-06-23T18:03:48.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- 172.21.15.22:0/1038447068 wait complete. 2023-06-23T18:03:48.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 Processor -- start 2023-06-23T18:03:48.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.544+0000 7f1d11189700 1 -- start start 2023-06-23T18:03:48.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0c070740 0x7f1d0c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158b20 0x7f1d0c15b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c1535c0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c1f4920 con 0x7f1d0c070740 2023-06-23T18:03:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c1f4c00 con 0x7f1d0c158b20 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d0bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0c070740 0x7f1d0c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47524/0 (socket says 172.21.15.22:47524) 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 -- 172.21.15.22:0/1857564007 learned_addr learned my addr 172.21.15.22:0/1857564007 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d037fe700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158b20 0x7f1d0c15b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 -- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158b20 msgr2=0x7f1d0c15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158b20 0x7f1d0c15b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 -- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0c070740 msgr2=0x7f1d0c153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0c070740 0x7f1d0c153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 -- 172.21.15.22:0/1857564007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009b30 con 0x7f1d0c158ef0 2023-06-23T18:03:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d10988700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c070220 tx=0x7f1d04004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d040099b0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d040092e0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c1f4ee0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04004470 con 0x7f1d0c158ef0 2023-06-23T18:03:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.548+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c1f54a0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c04e4c0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1d040045d0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d097fa700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 0x7f1cec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1d04084f60 con 0x7f1d0c158ef0 2023-06-23T18:03:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d0bfff700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 0x7f1cec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.552+0000 7f1d0bfff700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 0x7f1cec0687a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009b70 tx=0x7f1cfc009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.560+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1d04052100 con 0x7f1d0c158ef0 2023-06-23T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.728+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d0c06ebf0 con 0x7f1d0c158ef0 2023-06-23T18:03:48.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.728+0000 7f1d097fa700 1 -- 172.21.15.22:0/1857564007 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d04055720 con 0x7f1d0c158ef0 2023-06-23T18:03:48.741 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:03:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 msgr2=0x7f1cec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 0x7f1cec0687a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009b70 tx=0x7f1cfc009240 comp rx=0 tx=0).stop 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 msgr2=0x7f1d0c069960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c070220 tx=0x7f1d04004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 shutdown_connections 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c158ef0 0x7f1d0c069960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d0c158b20 0x7f1d0c15b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1cec0662f0 0x7f1cec0687a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 --2- 172.21.15.22:0/1857564007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0c070740 0x7f1d0c153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 >> 172.21.15.22:0/1857564007 conn(0x7f1d0c06b700 msgr2=0x7f1d0c0aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 shutdown_connections 2023-06-23T18:03:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.732+0000 7f1d11189700 1 -- 172.21.15.22:0/1857564007 wait complete. 2023-06-23T18:03:48.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:03:48.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:03:48.795 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:03:48.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 Processor -- start 2023-06-23T18:03:48.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 -- start start 2023-06-23T18:03:48.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fec7406e6e0 0x7fec7406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7415a0a0 con 0x7fec7406da30 2023-06-23T18:03:48.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7406f070 con 0x7fec7406e6e0 2023-06-23T18:03:48.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7ac81700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec740b1740 con 0x7fec74159cd0 2023-06-23T18:03:48.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fec7406e6e0 0x7fec7406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fec7406e6e0 0x7fec7406eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47530/0 (socket says 172.21.15.22:47530) 2023-06-23T18:03:48.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 -- 172.21.15.22:0/3174456613 learned_addr learned my addr 172.21.15.22:0/3174456613 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:48.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec79c7f700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 -- 172.21.15.22:0/3174456613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 msgr2=0x7fec740b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 -- 172.21.15.22:0/3174456613 >> v1:172.21.15.169:6789/0 conn(0x7fec7406da30 legacy=0x7fec740cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 -- 172.21.15.22:0/3174456613 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740cae40 con 0x7fec7406e6e0 2023-06-23T18:03:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec7947e700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406e6e0 0x7fec7406eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fec64009e50 tx=0x7fec6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a93cc3bb03dd563 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec78c7d700 1 -- 172.21.15.22:0/3174456613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400e070 con 0x7fec7406e6e0 2023-06-23T18:03:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec78c7d700 1 -- 172.21.15.22:0/3174456613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6400b9c0 con 0x7fec7406e6e0 2023-06-23T18:03:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.856+0000 7fec78c7d700 1 -- 172.21.15.22:0/3174456613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400bd70 con 0x7fec7406e6e0 2023-06-23T18:03:48.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- 172.21.15.22:0/3174456613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406e6e0 msgr2=0x7fec7406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406e6e0 0x7fec7406eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fec64009e50 tx=0x7fec6400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- 172.21.15.22:0/3174456613 shutdown_connections 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406e6e0 0x7fec7406eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3174456613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- 172.21.15.22:0/3174456613 >> 172.21.15.22:0/3174456613 conn(0x7fec7406b700 msgr2=0x7fec740b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- 172.21.15.22:0/3174456613 shutdown_connections 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- 172.21.15.22:0/3174456613 wait complete. 2023-06-23T18:03:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 Processor -- start 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.860+0000 7fec7ac81700 1 -- start start 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7406e6e0 0x7fec741537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec740c66d0 con 0x7fec7406da30 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec740cd040 con 0x7fec7406e6e0 2023-06-23T18:03:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec74153cf0 con 0x7fec74159cd0 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47542/0 (socket says 172.21.15.22:47542) 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 -- 172.21.15.22:0/3849912133 learned_addr learned my addr 172.21.15.22:0/3849912133 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7a480700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7947e700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7406e6e0 0x7fec741537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 -- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 msgr2=0x7fec740c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 -- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7406e6e0 msgr2=0x7fec741537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7406e6e0 0x7fec741537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 -- 172.21.15.22:0/3849912133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64009b30 con 0x7fec7406da30 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec79c7f700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d800 tx=0x7fec5c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0125f0 con 0x7fec7406da30 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c012750 con 0x7fec7406da30 2023-06-23T18:03:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec740cd2e0 con 0x7fec7406da30 2023-06-23T18:03:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012ac0 con 0x7fec7406da30 2023-06-23T18:03:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.864+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec740cd8a0 con 0x7fec7406da30 2023-06-23T18:03:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.868+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7404e4c0 con 0x7fec7406da30 2023-06-23T18:03:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.868+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fec5c012c20 con 0x7fec7406da30 2023-06-23T18:03:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.868+0000 7fec727fc700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 0x7fec540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:48.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.868+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec5c084130 con 0x7fec7406da30 2023-06-23T18:03:48.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.868+0000 7fec7947e700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 0x7fec540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:48.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.872+0000 7fec7947e700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 0x7fec540687a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec6400b370 tx=0x7fec64009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:48.876+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fec5c0512e0 con 0x7fec7406da30 2023-06-23T18:03:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.040+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fec740cb3d0 con 0x7fec7406da30 2023-06-23T18:03:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec727fc700 1 -- 172.21.15.22:0/3849912133 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec5c054900 con 0x7fec7406da30 2023-06-23T18:03:49.056 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:03:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 msgr2=0x7fec540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 0x7fec540687a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec6400b370 tx=0x7fec64009000 comp rx=0 tx=0).stop 2023-06-23T18:03:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 msgr2=0x7fec740c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d800 tx=0x7fec5c00db10 comp rx=0 tx=0).stop 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 shutdown_connections 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74159cd0 0x7fec740c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fec540662f0 0x7fec540687a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7406e6e0 0x7fec741537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 --2- 172.21.15.22:0/3849912133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec7406da30 0x7fec740c6190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 >> 172.21.15.22:0/3849912133 conn(0x7fec7406b700 msgr2=0x7fec741571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 shutdown_connections 2023-06-23T18:03:49.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.044+0000 7fec7ac81700 1 -- 172.21.15.22:0/3849912133 wait complete. 2023-06-23T18:03:49.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:03:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 Processor -- start 2023-06-23T18:03:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 -- start start 2023-06-23T18:03:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f52e406f660 0x7f52e4070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40712f0 0x7f52e415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e415a210 con 0x7f52e40afa90 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e415a020 con 0x7f52e406f660 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e9399700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e415a4f0 con 0x7f52e40712f0 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f52e406f660 0x7f52e4070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f52e406f660 0x7f52e4070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47552/0 (socket says 172.21.15.22:47552) 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 -- 172.21.15.22:0/925526103 learned_addr learned my addr 172.21.15.22:0/925526103 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e8b98700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40712f0 0x7f52e415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 -- 172.21.15.22:0/925526103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40712f0 msgr2=0x7f52e415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40712f0 0x7f52e415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 -- 172.21.15.22:0/925526103 >> v1:172.21.15.169:6789/0 conn(0x7f52e40afa90 legacy=0x7f52e40afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 -- 172.21.15.22:0/925526103 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e415a7d0 con 0x7f52e406f660 2023-06-23T18:03:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e37fe700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e406f660 0x7f52e4070db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2662ccbda7921bf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.220+0000 7f52e2ffd700 1 -- 172.21.15.22:0/925526103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400e070 con 0x7f52e406f660 2023-06-23T18:03:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e2ffd700 1 -- 172.21.15.22:0/925526103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4003a40 con 0x7f52e406f660 2023-06-23T18:03:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e2ffd700 1 -- 172.21.15.22:0/925526103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4003df0 con 0x7f52e406f660 2023-06-23T18:03:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- 172.21.15.22:0/925526103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e406f660 msgr2=0x7f52e4070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e406f660 0x7f52e4070db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).stop 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- 172.21.15.22:0/925526103 shutdown_connections 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40712f0 0x7f52e415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- 172.21.15.22:0/925526103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e406f660 0x7f52e4070db0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- 172.21.15.22:0/925526103 >> 172.21.15.22:0/925526103 conn(0x7f52e406a5f0 msgr2=0x7f52e40caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- 172.21.15.22:0/925526103 shutdown_connections 2023-06-23T18:03:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- 172.21.15.22:0/925526103 wait complete. 2023-06-23T18:03:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 Processor -- start 2023-06-23T18:03:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- start start 2023-06-23T18:03:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e406f660 0x7f52e40c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 0x7f52e40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c91d0 con 0x7f52e40712f0 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c6650 con 0x7f52e406f660 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e9399700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c6930 con 0x7f52e40afa90 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47560/0 (socket says 172.21.15.22:47560) 2023-06-23T18:03:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.224+0000 7f52e37fe700 1 -- 172.21.15.22:0/4226229050 learned_addr learned my addr 172.21.15.22:0/4226229050 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e8b98700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 0x7f52e40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e3fff700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e406f660 0x7f52e40c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 -- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 msgr2=0x7f52e40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 0x7f52e40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 -- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e406f660 msgr2=0x7f52e40c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e406f660 0x7f52e40c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 -- 172.21.15.22:0/4226229050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009b30 con 0x7f52e40712f0 2023-06-23T18:03:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e8b98700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 0x7f52e40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e37fe700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52d4015040 tx=0x7f52d4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40044c0 con 0x7f52e40712f0 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4004640 con 0x7f52e40712f0 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4012430 con 0x7f52e40712f0 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e40c6c10 con 0x7f52e40712f0 2023-06-23T18:03:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e40c7670 con 0x7f52e40712f0 2023-06-23T18:03:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e40c6e30 con 0x7f52e40712f0 2023-06-23T18:03:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.228+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f52d4012590 con 0x7f52e40712f0 2023-06-23T18:03:49.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.232+0000 7f52e0ff9700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 0x7f52d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.232+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f52d4085580 con 0x7f52e40712f0 2023-06-23T18:03:49.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.232+0000 7f52e3fff700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 0x7f52d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.232+0000 7f52e3fff700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 0x7f52d00687a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f52d8004f70 tx=0x7f52d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.236+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f52d4052720 con 0x7f52e40712f0 2023-06-23T18:03:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.416+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52e404e4c0 con 0x7f52e40712f0 2023-06-23T18:03:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.420+0000 7f52e0ff9700 1 -- 172.21.15.22:0/4226229050 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52d4055d40 con 0x7f52e40712f0 2023-06-23T18:03:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 msgr2=0x7f52d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 0x7f52d00687a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f52d8004f70 tx=0x7f52d8009040 comp rx=0 tx=0).stop 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 msgr2=0x7f52e4150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52d4015040 tx=0x7f52d4004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 shutdown_connections 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e40afa90 0x7f52e40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52e40712f0 0x7f52e4150870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f52d00662f0 0x7f52d00687a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 --2- 172.21.15.22:0/4226229050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e406f660 0x7f52e40c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 >> 172.21.15.22:0/4226229050 conn(0x7f52e406a5f0 msgr2=0x7f52e40caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 shutdown_connections 2023-06-23T18:03:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.424+0000 7f52e9399700 1 -- 172.21.15.22:0/4226229050 wait complete. 2023-06-23T18:03:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:03:49.452 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:03:49.452 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:03:49.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 Processor -- start 2023-06-23T18:03:49.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- start start 2023-06-23T18:03:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00160420 0x7f7a000cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7a00070040 0x7f7a000cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a00163ec0 con 0x7f7a000c0fe0 2023-06-23T18:03:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a000cc6f0 con 0x7f7a00070040 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001641a0 con 0x7f7a00160420 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79feffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7a000c0fe0 0x7f7a000c13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57116/0 (socket says 172.21.15.22:57116) 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79feffd700 1 -- 172.21.15.22:0/784747483 learned_addr learned my addr 172.21.15.22:0/784747483 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/784747483 >> v2:172.21.15.22:3300/0 conn(0x7f7a00070040 0x7f7a000cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fe7fc700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00160420 0x7f7a000cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fdffb700 1 -- 172.21.15.22:0/784747483 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871011177 0 0) 0x7f7a00163ec0 con 0x7f7a000c0fe0 2023-06-23T18:03:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fdffb700 1 -- 172.21.15.22:0/784747483 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec0035d0 con 0x7f7a000c0fe0 2023-06-23T18:03:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 -- 172.21.15.22:0/784747483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00160420 msgr2=0x7f7a000cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00160420 0x7f7a000cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 -- 172.21.15.22:0/784747483 >> v1:172.21.15.22:6789/0 conn(0x7f7a000c0fe0 legacy=0x7f7a000c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 -- 172.21.15.22:0/784747483 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00164480 con 0x7f7a00070040 2023-06-23T18:03:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00070040 0x7f7a000cc1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009850 tx=0x7f79f0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f10984a78bb8d25 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fdffb700 1 -- 172.21.15.22:0/784747483 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011830 con 0x7f7a00070040 2023-06-23T18:03:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fdffb700 1 -- 172.21.15.22:0/784747483 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0011990 con 0x7f7a00070040 2023-06-23T18:03:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fdffb700 1 -- 172.21.15.22:0/784747483 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011d40 con 0x7f7a00070040 2023-06-23T18:03:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- 172.21.15.22:0/784747483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00070040 msgr2=0x7f7a000cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00070040 0x7f7a000cc1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009850 tx=0x7f79f0009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- 172.21.15.22:0/784747483 shutdown_connections 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00070040 0x7f7a000cc1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 --2- 172.21.15.22:0/784747483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00160420 0x7f7a000cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- 172.21.15.22:0/784747483 >> 172.21.15.22:0/784747483 conn(0x7f7a0006a5f0 msgr2=0x7f7a000caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.556+0000 7f79fffff700 1 -- 172.21.15.22:0/784747483 shutdown_connections 2023-06-23T18:03:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.560+0000 7f79fffff700 1 -- 172.21.15.22:0/784747483 wait complete. 2023-06-23T18:03:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.560+0000 7f79fffff700 1 Processor -- start 2023-06-23T18:03:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.560+0000 7f79fffff700 1 -- start start 2023-06-23T18:03:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070040 0x7f7a000b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 0x7f7a001f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001f6be0 con 0x7f7a00160420 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001f9290 con 0x7f7a000c0fe0 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001f9530 con 0x7f7a00070040 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47586/0 (socket says 172.21.15.22:47586) 2023-06-23T18:03:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 -- 172.21.15.22:0/2004694167 learned_addr learned my addr 172.21.15.22:0/2004694167 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:49.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fe7fc700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 0x7f7a001f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79feffd700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070040 0x7f7a000b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 -- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070040 msgr2=0x7f7a000b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070040 0x7f7a000b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 -- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 msgr2=0x7f7a001f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 0x7f7a001f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 -- 172.21.15.22:0/2004694167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00094b0 con 0x7f7a00160420 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fe7fc700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 0x7f7a001f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79ff7fe700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7a0006dd50 tx=0x7f79f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0004380 con 0x7f7a00160420 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f001f070 con 0x7f7a00160420 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a001f9810 con 0x7f7a00160420 2023-06-23T18:03:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00036c0 con 0x7f7a00160420 2023-06-23T18:03:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.564+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a001f6870 con 0x7f7a00160420 2023-06-23T18:03:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a00164740 con 0x7f7a00160420 2023-06-23T18:03:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f79f0021030 con 0x7f7a00160420 2023-06-23T18:03:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79f77fe700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 0x7f79e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f79f0085230 con 0x7f7a00160420 2023-06-23T18:03:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79feffd700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 0x7f79e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.568+0000 7f79feffd700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 0x7f79e00687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007fd0 tx=0x7f79e8007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.576+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7a00164740 con 0x7f7a00160420 2023-06-23T18:03:49.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.736+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a00164740 con 0x7f7a00160420 2023-06-23T18:03:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.740+0000 7f79f77fe700 1 -- 172.21.15.22:0/2004694167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f7a00164740 con 0x7f7a00160420 2023-06-23T18:03:49.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:49.755 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 msgr2=0x7f79e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 0x7f79e00687a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007fd0 tx=0x7f79e8007de0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 msgr2=0x7f7a0015cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7a0006dd50 tx=0x7f79f0004770 comp rx=0 tx=0).stop 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 shutdown_connections 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a00160420 0x7f7a0015cdb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79e00662f0 0x7f79e00687a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000c0fe0 0x7f7a001f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 --2- 172.21.15.22:0/2004694167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070040 0x7f7a000b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 >> 172.21.15.22:0/2004694167 conn(0x7f7a0006a5f0 msgr2=0x7f7a0006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 shutdown_connections 2023-06-23T18:03:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.744+0000 7f79fffff700 1 -- 172.21.15.22:0/2004694167 wait complete. 2023-06-23T18:03:49.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:03:49.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:03:49.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.884+0000 7f4cfa29e700 1 Processor -- start 2023-06-23T18:03:49.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.884+0000 7f4cfa29e700 1 -- start start 2023-06-23T18:03:49.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4cf40b4730 0x7f4cf40d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf40b53c0 0x7f4cf40b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf40d0420 con 0x7f4cf415f3b0 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf40b5d40 con 0x7f4cf40b4730 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40cd670 con 0x7f4cf40b53c0 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf3fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4cf415f3b0 0x7f4cf40ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57126/0 (socket says 172.21.15.22:57126) 2023-06-23T18:03:49.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf3fff700 1 -- 172.21.15.22:0/1116958156 learned_addr learned my addr 172.21.15.22:0/1116958156 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:49.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/1116958156 >> v2:172.21.15.169:3300/0 conn(0x7f4cf40b4730 0x7f4cf40d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf883b700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf40b53c0 0x7f4cf40b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf2ffd700 1 -- 172.21.15.22:0/1116958156 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362134453 0 0) 0x7f4cf40d0420 con 0x7f4cf415f3b0 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf2ffd700 1 -- 172.21.15.22:0/1116958156 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cdc0035d0 con 0x7f4cf415f3b0 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 -- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf40b53c0 msgr2=0x7f4cf40b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf40b53c0 0x7f4cf40b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 -- 172.21.15.22:0/1116958156 >> v1:172.21.15.22:6789/0 conn(0x7f4cf415f3b0 legacy=0x7f4cf40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 -- 172.21.15.22:0/1116958156 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40cd950 con 0x7f4cf40b4730 2023-06-23T18:03:49.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf40d1cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009850 tx=0x7f4ce8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74782e1aaa56a2df server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf2ffd700 1 -- 172.21.15.22:0/1116958156 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8011830 con 0x7f4cf40b4730 2023-06-23T18:03:49.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf2ffd700 1 -- 172.21.15.22:0/1116958156 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8011990 con 0x7f4cf40b4730 2023-06-23T18:03:49.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cf2ffd700 1 -- 172.21.15.22:0/1116958156 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8011d40 con 0x7f4cf40b4730 2023-06-23T18:03:49.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 msgr2=0x7f4cf40d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf40d1cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009850 tx=0x7f4ce8009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:49.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- 172.21.15.22:0/1116958156 shutdown_connections 2023-06-23T18:03:49.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf40b53c0 0x7f4cf40b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/1116958156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf40d1cb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- 172.21.15.22:0/1116958156 >> 172.21.15.22:0/1116958156 conn(0x7f4cf4069960 msgr2=0x7f4cf4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- 172.21.15.22:0/1116958156 shutdown_connections 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.888+0000 7f4cfa29e700 1 -- 172.21.15.22:0/1116958156 wait complete. 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 Processor -- start 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- start start 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf415f3b0 0x7f4cf415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4154ab0 con 0x7f4cf40b53c0 2023-06-23T18:03:49.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40caf90 con 0x7f4cf40b4730 2023-06-23T18:03:49.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf41f5730 con 0x7f4cf415f3b0 2023-06-23T18:03:49.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47600/0 (socket says 172.21.15.22:47600) 2023-06-23T18:03:49.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 -- 172.21.15.22:0/3933201217 learned_addr learned my addr 172.21.15.22:0/3933201217 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf883b700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf415f3b0 0x7f4cf415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf3fff700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 -- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf415f3b0 msgr2=0x7f4cf415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf415f3b0 0x7f4cf415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 -- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 msgr2=0x7f4cf4154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf4154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:49.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 -- 172.21.15.22:0/3933201217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80094b0 con 0x7f4cf40b53c0 2023-06-23T18:03:49.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf37fe700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8014040 tx=0x7f4ce8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80036c0 con 0x7f4cf40b53c0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce801e070 con 0x7f4cf40b53c0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003a80 con 0x7f4cf40b53c0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf41f5170 con 0x7f4cf40b53c0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.892+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf40cac20 con 0x7f4cf40b53c0 2023-06-23T18:03:49.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf40cb140 con 0x7f4cf40b53c0 2023-06-23T18:03:49.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4ce8003be0 con 0x7f4cf40b53c0 2023-06-23T18:03:49.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cf0ff9700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 0x7f4ce00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:49.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cf3fff700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 0x7f4ce00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:49.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ce8020070 con 0x7f4cf40b53c0 2023-06-23T18:03:49.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.896+0000 7f4cf3fff700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 0x7f4ce00687a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40011e0 tx=0x7f4ce4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:49.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:49.904+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4cf40cb140 con 0x7f4cf40b53c0 2023-06-23T18:03:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.068+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cf40cb100 con 0x7f4cf40b53c0 2023-06-23T18:03:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.068+0000 7f4cf0ff9700 1 -- 172.21.15.22:0/3933201217 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ce8052620 con 0x7f4cf40b53c0 2023-06-23T18:03:50.083 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:03:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 msgr2=0x7f4ce00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 0x7f4ce00687a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40011e0 tx=0x7f4ce4009040 comp rx=0 tx=0).stop 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 msgr2=0x7f4cf41f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8014040 tx=0x7f4ce8004770 comp rx=0 tx=0).stop 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 shutdown_connections 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf415f3b0 0x7f4cf415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cf40b53c0 0x7f4cf41f4800 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4ce00662f0 0x7f4ce00687a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 --2- 172.21.15.22:0/3933201217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40b4730 0x7f4cf4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 >> 172.21.15.22:0/3933201217 conn(0x7f4cf4069960 msgr2=0x7f4cf40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 shutdown_connections 2023-06-23T18:03:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.072+0000 7f4cfa29e700 1 -- 172.21.15.22:0/3933201217 wait complete. 2023-06-23T18:03:50.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:03:50.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 Processor -- start 2023-06-23T18:03:50.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 -- start start 2023-06-23T18:03:50.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d440e2af0 0x7f5d4403adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d44006e60 0x7f5d44037000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d44037540 0x7f5d44050e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d440379c0 con 0x7f5d44037540 2023-06-23T18:03:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d44008090 con 0x7f5d44006e60 2023-06-23T18:03:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.208+0000 7f5d52e74700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44007690 con 0x7f5d440e2af0 2023-06-23T18:03:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d44037540 0x7f5d44050e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d44037540 0x7f5d44050e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47612/0 (socket says 172.21.15.22:47612) 2023-06-23T18:03:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 -- 172.21.15.22:0/735299083 learned_addr learned my addr 172.21.15.22:0/735299083 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d51671700 1 --2- 172.21.15.22:0/735299083 >> v2:172.21.15.169:3300/0 conn(0x7f5d44006e60 0x7f5d44037000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d51e72700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d440e2af0 0x7f5d4403adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 -- 172.21.15.22:0/735299083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d440e2af0 msgr2=0x7f5d4403adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d440e2af0 0x7f5d4403adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 -- 172.21.15.22:0/735299083 >> v2:172.21.15.169:3300/0 conn(0x7f5d44006e60 msgr2=0x7f5d44037000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:50.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 --2- 172.21.15.22:0/735299083 >> v2:172.21.15.169:3300/0 conn(0x7f5d44006e60 0x7f5d44037000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 -- 172.21.15.22:0/735299083 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44007970 con 0x7f5d44037540 2023-06-23T18:03:50.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52673700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d44037540 0x7f5d44050e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c078870 tx=0x7f5d4c0674a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e5d03905ff25bd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d50e70700 1 -- 172.21.15.22:0/735299083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c073810 con 0x7f5d44037540 2023-06-23T18:03:50.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d50e70700 1 -- 172.21.15.22:0/735299083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c073970 con 0x7f5d44037540 2023-06-23T18:03:50.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d50e70700 1 -- 172.21.15.22:0/735299083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c06da30 con 0x7f5d44037540 2023-06-23T18:03:50.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 -- 172.21.15.22:0/735299083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d44037540 msgr2=0x7f5d44050e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d44037540 0x7f5d44050e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c078870 tx=0x7f5d4c0674a0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 -- 172.21.15.22:0/735299083 shutdown_connections 2023-06-23T18:03:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d44037540 0x7f5d44050e60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 --2- 172.21.15.22:0/735299083 >> v2:172.21.15.169:3300/0 conn(0x7f5d44006e60 0x7f5d44037000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 --2- 172.21.15.22:0/735299083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d440e2af0 0x7f5d4403adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 -- 172.21.15.22:0/735299083 >> 172.21.15.22:0/735299083 conn(0x7f5d44052000 msgr2=0x7f5d44052400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 -- 172.21.15.22:0/735299083 shutdown_connections 2023-06-23T18:03:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.212+0000 7f5d52e74700 1 -- 172.21.15.22:0/735299083 wait complete. 2023-06-23T18:03:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 Processor -- start 2023-06-23T18:03:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- start start 2023-06-23T18:03:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d44006e60 0x7f5d4404ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d44037540 0x7f5d440de660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4404c910 con 0x7f5d440e2af0 2023-06-23T18:03:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44049a00 con 0x7f5d44006e60 2023-06-23T18:03:50.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44177be0 con 0x7f5d44037540 2023-06-23T18:03:50.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47628/0 (socket says 172.21.15.22:47628) 2023-06-23T18:03:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 -- 172.21.15.22:0/18719274 learned_addr learned my addr 172.21.15.22:0/18719274 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d51e72700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d44006e60 0x7f5d4404ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d51671700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d44037540 0x7f5d440de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 -- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d44037540 msgr2=0x7f5d440de660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d44037540 0x7f5d440de660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 -- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d44006e60 msgr2=0x7f5d4404ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d44006e60 0x7f5d4404ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 -- 172.21.15.22:0/18719274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4404c5c0 con 0x7f5d440e2af0 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52673700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c068770 tx=0x7f5d4c0683f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c06d2e0 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c06d440 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d440deba0 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c04e590 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.216+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d44049f60 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4404cac0 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5d4c072d70 con 0x7f5d440e2af0 2023-06-23T18:03:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d427fc700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 0x7f5d300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d4c0fdd20 con 0x7f5d440e2af0 2023-06-23T18:03:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d51e72700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 0x7f5d300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.220+0000 7f5d51e72700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 0x7f5d300687a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00b9d0 tx=0x7f5d3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.228+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5d4404cac0 con 0x7f5d440e2af0 2023-06-23T18:03:50.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d4404cac0 con 0x7f5d440e2af0 2023-06-23T18:03:50.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d427fc700 1 -- 172.21.15.22:0/18719274 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d4404cac0 con 0x7f5d440e2af0 2023-06-23T18:03:50.415 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:03:50.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 msgr2=0x7f5d300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 0x7f5d300687a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00b9d0 tx=0x7f5d3c008040 comp rx=0 tx=0).stop 2023-06-23T18:03:50.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 msgr2=0x7f5d4417b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.400+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c068770 tx=0x7f5d4c0683f0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 shutdown_connections 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d440e2af0 0x7f5d4417b350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d44037540 0x7f5d440de660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5d300662f0 0x7f5d300687a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 --2- 172.21.15.22:0/18719274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d44006e60 0x7f5d4404ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 >> 172.21.15.22:0/18719274 conn(0x7f5d44052000 msgr2=0x7f5d440369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 shutdown_connections 2023-06-23T18:03:50.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.404+0000 7f5d52e74700 1 -- 172.21.15.22:0/18719274 wait complete. 2023-06-23T18:03:50.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:03:50.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 Processor -- start 2023-06-23T18:03:50.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 -- start start 2023-06-23T18:03:50.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c780707b0 0x7f0c780c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780cbaa0 con 0x7f0c78069960 2023-06-23T18:03:50.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780cbd80 con 0x7f0c78070c10 2023-06-23T18:03:50.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7f2a2700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c78071110 con 0x7f0c780707b0 2023-06-23T18:03:50.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7e2a0700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0c78069960 0x7f0c78069d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57128/0 (socket says 172.21.15.22:57128) 2023-06-23T18:03:50.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7e2a0700 1 -- 172.21.15.22:0/2689944829 learned_addr learned my addr 172.21.15.22:0/2689944829 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.524+0000 7f0c7da9f700 1 --2- 172.21.15.22:0/2689944829 >> v2:172.21.15.169:3300/0 conn(0x7f0c780707b0 0x7f0c780c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7d29e700 1 -- 172.21.15.22:0/2689944829 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950375546 0 0) 0x7f0c780cbaa0 con 0x7f0c78069960 2023-06-23T18:03:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7da9f700 1 -- 172.21.15.22:0/2689944829 >> v1:172.21.15.169:6789/0 conn(0x7f0c78070c10 legacy=0x7f0c780c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7da9f700 1 -- 172.21.15.22:0/2689944829 >> v1:172.21.15.22:6789/0 conn(0x7f0c78069960 legacy=0x7f0c78069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7da9f700 1 -- 172.21.15.22:0/2689944829 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780cc060 con 0x7f0c780707b0 2023-06-23T18:03:50.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7da9f700 1 --2- 172.21.15.22:0/2689944829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c780707b0 0x7f0c780c9710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009850 tx=0x7f0c68009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b9489d55b9ac2b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7d29e700 1 -- 172.21.15.22:0/2689944829 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68011830 con 0x7f0c780707b0 2023-06-23T18:03:50.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7d29e700 1 -- 172.21.15.22:0/2689944829 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68011990 con 0x7f0c780707b0 2023-06-23T18:03:50.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7d29e700 1 -- 172.21.15.22:0/2689944829 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68011d20 con 0x7f0c780707b0 2023-06-23T18:03:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2689944829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c780707b0 msgr2=0x7f0c780c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2689944829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c780707b0 0x7f0c780c9710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009850 tx=0x7f0c68009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2689944829 shutdown_connections 2023-06-23T18:03:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2689944829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c780707b0 0x7f0c780c9710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2689944829 >> 172.21.15.22:0/2689944829 conn(0x7f0c7806b700 msgr2=0x7f0c780b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2689944829 shutdown_connections 2023-06-23T18:03:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.528+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2689944829 wait complete. 2023-06-23T18:03:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 Processor -- start 2023-06-23T18:03:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- start start 2023-06-23T18:03:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 0x7f0c780b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c78070c10 0x7f0c78159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781597c0 con 0x7f0c780c8c70 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780b2670 con 0x7f0c78070c10 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780b2950 con 0x7f0c78069960 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47644/0 (socket says 172.21.15.22:47644) 2023-06-23T18:03:50.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7e2a0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 0x7f0c780b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 -- 172.21.15.22:0/2391702809 learned_addr learned my addr 172.21.15.22:0/2391702809 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:50.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7da9f700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c78070c10 0x7f0c78159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 -- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 msgr2=0x7f0c780b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 0x7f0c780b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 -- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c78070c10 msgr2=0x7f0c78159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c78070c10 0x7f0c78159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 -- 172.21.15.22:0/2391702809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680094b0 con 0x7f0c780c8c70 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7e2a0700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 0x7f0c780b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7eaa1700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000cc50 tx=0x7f0c7000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70003c20 con 0x7f0c780c8c70 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70003d80 con 0x7f0c780c8c70 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c780b2c30 con 0x7f0c780c8c70 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000aab0 con 0x7f0c780c8c70 2023-06-23T18:03:50.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.532+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c780b31f0 con 0x7f0c780c8c70 2023-06-23T18:03:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c780b33e0 con 0x7f0c780c8c70 2023-06-23T18:03:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0c7000ac10 con 0x7f0c780c8c70 2023-06-23T18:03:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c6effd700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 0x7f0c5c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c7e2a0700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 0x7f0c5c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0c70083fb0 con 0x7f0c780c8c70 2023-06-23T18:03:50.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.536+0000 7f0c7e2a0700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 0x7f0c5c0687a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007c60 tx=0x7f0c74007bb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.544+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0c70051130 con 0x7f0c780c8c70 2023-06-23T18:03:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.712+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c7804e4c0 con 0x7f0c780c8c70 2023-06-23T18:03:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.712+0000 7f0c6effd700 1 -- 172.21.15.22:0/2391702809 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c70054750 con 0x7f0c780c8c70 2023-06-23T18:03:50.726 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:03:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 msgr2=0x7f0c5c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 0x7f0c5c0687a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007c60 tx=0x7f0c74007bb0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 msgr2=0x7f0c780c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000cc50 tx=0x7f0c7000cf60 comp rx=0 tx=0).stop 2023-06-23T18:03:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 shutdown_connections 2023-06-23T18:03:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c780c8c70 0x7f0c780c6a10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c5c0662f0 0x7f0c5c0687a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c78070c10 0x7f0c78159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 --2- 172.21.15.22:0/2391702809 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c78069960 0x7f0c780b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 >> 172.21.15.22:0/2391702809 conn(0x7f0c7806b700 msgr2=0x7f0c780cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 shutdown_connections 2023-06-23T18:03:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.716+0000 7f0c7f2a2700 1 -- 172.21.15.22:0/2391702809 wait complete. 2023-06-23T18:03:50.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:03:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.852+0000 7f151521e700 1 Processor -- start 2023-06-23T18:03:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.852+0000 7f151521e700 1 -- start start 2023-06-23T18:03:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15100d2360 0x7f15100d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100ccb30 con 0x7f15100cf3a0 2023-06-23T18:03:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100d2880 con 0x7f15100d2360 2023-06-23T18:03:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100d2b60 con 0x7f15100d1f30 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15100d2360 0x7f15100d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15100d2360 0x7f15100d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47660/0 (socket says 172.21.15.22:47660) 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 -- 172.21.15.22:0/635180444 learned_addr learned my addr 172.21.15.22:0/635180444 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f150f7fe700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 -- 172.21.15.22:0/635180444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 msgr2=0x7f15100d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 -- 172.21.15.22:0/635180444 >> v1:172.21.15.169:6789/0 conn(0x7f15100cf3a0 legacy=0x7f15100cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 -- 172.21.15.22:0/635180444 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100d2e40 con 0x7f15100d2360 2023-06-23T18:03:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f1514a1d700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100d0560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1508009e50 tx=0x7f150800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd15ba9cc6df255 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f150effd700 1 -- 172.21.15.22:0/635180444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800e070 con 0x7f15100d2360 2023-06-23T18:03:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f150effd700 1 -- 172.21.15.22:0/635180444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800b910 con 0x7f15100d2360 2023-06-23T18:03:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f150effd700 1 -- 172.21.15.22:0/635180444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800bcc0 con 0x7f15100d2360 2023-06-23T18:03:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- 172.21.15.22:0/635180444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 msgr2=0x7f15100d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100d0560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1508009e50 tx=0x7f150800b330 comp rx=0 tx=0).stop 2023-06-23T18:03:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- 172.21.15.22:0/635180444 shutdown_connections 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100d0560 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 --2- 172.21.15.22:0/635180444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- 172.21.15.22:0/635180444 >> 172.21.15.22:0/635180444 conn(0x7f151006a280 msgr2=0x7f151009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- 172.21.15.22:0/635180444 shutdown_connections 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.856+0000 7f151521e700 1 -- 172.21.15.22:0/635180444 wait complete. 2023-06-23T18:03:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.860+0000 7f151521e700 1 Processor -- start 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.860+0000 7f151521e700 1 -- start start 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15100cf3a0 0x7f1510159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b5400 con 0x7f15100d2360 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b56e0 con 0x7f15100cf3a0 2023-06-23T18:03:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b59c0 con 0x7f15100d1f30 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47664/0 (socket says 172.21.15.22:47664) 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f150ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15100cf3a0 0x7f1510159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 -- 172.21.15.22:0/3284347790 learned_addr learned my addr 172.21.15.22:0/3284347790 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f150f7fe700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 -- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 msgr2=0x7f15100b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 -- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15100cf3a0 msgr2=0x7f1510159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15100cf3a0 0x7f1510159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 -- 172.21.15.22:0/3284347790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508009b30 con 0x7f15100d2360 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f1514a1d700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1510066d40 tx=0x7f1508004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508004800 con 0x7f15100d2360 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15080099b0 con 0x7f15100d2360 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15100b5ca0 con 0x7f15100d2360 2023-06-23T18:03:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150801c680 con 0x7f15100d2360 2023-06-23T18:03:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.864+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15100b7dd0 con 0x7f15100d2360 2023-06-23T18:03:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151006c2f0 con 0x7f15100d2360 2023-06-23T18:03:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f150801c7e0 con 0x7f15100d2360 2023-06-23T18:03:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f150cff9700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 0x7f14f00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1508085790 con 0x7f15100d2360 2023-06-23T18:03:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f150ffff700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 0x7f14f00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.868+0000 7f150ffff700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 0x7f14f00687a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14f8000fb0 tx=0x7f14f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:50.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:50.876+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1508052920 con 0x7f15100d2360 2023-06-23T18:03:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.032+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15100b7f60 con 0x7f15100d2360 2023-06-23T18:03:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f150cff9700 1 -- 172.21.15.22:0/3284347790 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1508055f40 con 0x7f15100d2360 2023-06-23T18:03:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:03:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 msgr2=0x7f14f00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 0x7f14f00687a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14f8000fb0 tx=0x7f14f8009040 comp rx=0 tx=0).stop 2023-06-23T18:03:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 msgr2=0x7f15100cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1510066d40 tx=0x7f1508004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 shutdown_connections 2023-06-23T18:03:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15100d2360 0x7f15100cd580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100d1f30 0x7f15100b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14f00662f0 0x7f14f00687a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 --2- 172.21.15.22:0/3284347790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15100cf3a0 0x7f1510159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 >> 172.21.15.22:0/3284347790 conn(0x7f151006a280 msgr2=0x7f151015b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 shutdown_connections 2023-06-23T18:03:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.036+0000 7f151521e700 1 -- 172.21.15.22:0/3284347790 wait complete. 2023-06-23T18:03:51.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:03:51.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.160+0000 7f18ebfff700 1 Processor -- start 2023-06-23T18:03:51.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.160+0000 7f18ebfff700 1 -- start start 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18ec0ad070 0x7f18ec0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0c1040 0x7f18ec0c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0affc0 con 0x7f18ec160480 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0c1a00 con 0x7f18ec0ad070 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0ad950 con 0x7f18ec0c1040 2023-06-23T18:03:51.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18eaffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f18ec160480 0x7f18ec0b0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57140/0 (socket says 172.21.15.22:57140) 2023-06-23T18:03:51.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18eaffd700 1 -- 172.21.15.22:0/2451337606 learned_addr learned my addr 172.21.15.22:0/2451337606 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:51.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2451337606 >> v2:172.21.15.22:3300/0 conn(0x7f18ec0ad070 0x7f18ec0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18eb7fe700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0c1040 0x7f18ec0c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ea7fc700 1 -- 172.21.15.22:0/2451337606 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230051149 0 0) 0x7f18ec0affc0 con 0x7f18ec160480 2023-06-23T18:03:51.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ea7fc700 1 -- 172.21.15.22:0/2451337606 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d80035d0 con 0x7f18ec160480 2023-06-23T18:03:51.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 -- 172.21.15.22:0/2451337606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0c1040 msgr2=0x7f18ec0c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:51.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0c1040 0x7f18ec0c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 -- 172.21.15.22:0/2451337606 >> v1:172.21.15.22:6789/0 conn(0x7f18ec160480 legacy=0x7f18ec0b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 -- 172.21.15.22:0/2451337606 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0afc90 con 0x7f18ec0ad070 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0ad070 0x7f18ec0ccf50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009850 tx=0x7f18d4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8f11516b3a13d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ea7fc700 1 -- 172.21.15.22:0/2451337606 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18ec0ad070 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ea7fc700 1 -- 172.21.15.22:0/2451337606 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4011990 con 0x7f18ec0ad070 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ea7fc700 1 -- 172.21.15.22:0/2451337606 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011d00 con 0x7f18ec0ad070 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- 172.21.15.22:0/2451337606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0ad070 msgr2=0x7f18ec0ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0ad070 0x7f18ec0ccf50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009850 tx=0x7f18d4009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- 172.21.15.22:0/2451337606 shutdown_connections 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0c1040 0x7f18ec0c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2451337606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0ad070 0x7f18ec0ccf50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- 172.21.15.22:0/2451337606 >> 172.21.15.22:0/2451337606 conn(0x7f18ec06a5f0 msgr2=0x7f18ec0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- 172.21.15.22:0/2451337606 shutdown_connections 2023-06-23T18:03:51.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.164+0000 7f18ebfff700 1 -- 172.21.15.22:0/2451337606 wait complete. 2023-06-23T18:03:51.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 Processor -- start 2023-06-23T18:03:51.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 -- start start 2023-06-23T18:03:51.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0ad070 0x7f18ec1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ec160480 0x7f18ec0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1fa3b0 con 0x7f18ec0c1040 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1f94e0 con 0x7f18ec160480 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec151720 con 0x7f18ec0ad070 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47676/0 (socket says 172.21.15.22:47676) 2023-06-23T18:03:51.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 -- 172.21.15.22:0/2221310267 learned_addr learned my addr 172.21.15.22:0/2221310267 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:51.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18eaffd700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0ad070 0x7f18ec1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18eb7fe700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ec160480 0x7f18ec0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 -- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0ad070 msgr2=0x7f18ec1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0ad070 0x7f18ec1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 -- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ec160480 msgr2=0x7f18ec0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ec160480 0x7f18ec0c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 -- 172.21.15.22:0/2221310267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40094b0 con 0x7f18ec0c1040 2023-06-23T18:03:51.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e3fff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18d40090b0 tx=0x7f18d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4004590 con 0x7f18ec0c1040 2023-06-23T18:03:51.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d401f070 con 0x7f18ec0c1040 2023-06-23T18:03:51.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.168+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec1f9730 con 0x7f18ec0c1040 2023-06-23T18:03:51.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003900 con 0x7f18ec0c1040 2023-06-23T18:03:51.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec1f9cf0 con 0x7f18ec0c1040 2023-06-23T18:03:51.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec04e4c0 con 0x7f18ec0c1040 2023-06-23T18:03:51.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f18d4003a60 con 0x7f18ec0c1040 2023-06-23T18:03:51.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18e37fe700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 0x7f18cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f18d4021070 con 0x7f18ec0c1040 2023-06-23T18:03:51.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.172+0000 7f18eaffd700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 0x7f18cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.176+0000 7f18eaffd700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 0x7f18cc0687a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f18dc007fd0 tx=0x7f18dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.184+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f18d4052880 con 0x7f18ec0c1040 2023-06-23T18:03:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.348+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f18ec0af9d0 con 0x7f18ec0c1040 2023-06-23T18:03:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.348+0000 7f18e37fe700 1 -- 172.21.15.22:0/2221310267 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f18d4055ea0 con 0x7f18ec0c1040 2023-06-23T18:03:51.362 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:03:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 msgr2=0x7f18cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 0x7f18cc0687a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f18dc007fd0 tx=0x7f18dc009040 comp rx=0 tx=0).stop 2023-06-23T18:03:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 msgr2=0x7f18ec0c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18d40090b0 tx=0x7f18d4004770 comp rx=0 tx=0).stop 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 shutdown_connections 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f18cc0662f0 0x7f18cc0687a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ec160480 0x7f18ec0c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec0c1040 0x7f18ec0c94b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 --2- 172.21.15.22:0/2221310267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ec0ad070 0x7f18ec1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 >> 172.21.15.22:0/2221310267 conn(0x7f18ec06a5f0 msgr2=0x7f18ec15f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 shutdown_connections 2023-06-23T18:03:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.352+0000 7f18ebfff700 1 -- 172.21.15.22:0/2221310267 wait complete. 2023-06-23T18:03:51.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:03:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.476+0000 7fcce33f9700 1 Processor -- start 2023-06-23T18:03:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- start start 2023-06-23T18:03:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 0x7fccdc0c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccdc0c18f0 con 0x7fccdc160420 2023-06-23T18:03:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccdc163ec0 con 0x7fccdc0cc380 2023-06-23T18:03:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1641a0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce1bf6700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fccdc160420 0x7fccdc0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57144/0 (socket says 172.21.15.22:57144) 2023-06-23T18:03:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce1bf6700 1 -- 172.21.15.22:0/3938872135 learned_addr learned my addr 172.21.15.22:0/3938872135 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce23f7700 1 --2- 172.21.15.22:0/3938872135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 0x7fccdc0c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442011368 0 0) 0x7fccdc0c18f0 con 0x7fccdc160420 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc80035d0 con 0x7fccdc160420 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655480877 0 0) 0x7fccdc163ec0 con 0x7fccdc0cc380 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc0c18f0 con 0x7fccdc0cc380 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce23f7700 1 -- 172.21.15.22:0/3938872135 >> v1:172.21.15.169:6789/0 conn(0x7fccdc0cc380 legacy=0x7fccdc0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce23f7700 1 -- 172.21.15.22:0/3938872135 >> v1:172.21.15.22:6789/0 conn(0x7fccdc160420 legacy=0x7fccdc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce23f7700 1 -- 172.21.15.22:0/3938872135 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc164480 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce23f7700 1 --2- 172.21.15.22:0/3938872135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 0x7fccdc0c13b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009dc0 tx=0x7fccd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15056898d816de71 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000e070 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd000bb20 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce13f5700 1 -- 172.21.15.22:0/3938872135 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0012430 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- 172.21.15.22:0/3938872135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 msgr2=0x7fccdc0c13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 --2- 172.21.15.22:0/3938872135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 0x7fccdc0c13b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009dc0 tx=0x7fccd000b330 comp rx=0 tx=0).stop 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- 172.21.15.22:0/3938872135 shutdown_connections 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 --2- 172.21.15.22:0/3938872135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0c0fe0 0x7fccdc0c13b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- 172.21.15.22:0/3938872135 >> 172.21.15.22:0/3938872135 conn(0x7fccdc06a5f0 msgr2=0x7fccdc0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- 172.21.15.22:0/3938872135 shutdown_connections 2023-06-23T18:03:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.480+0000 7fcce33f9700 1 -- 172.21.15.22:0/3938872135 wait complete. 2023-06-23T18:03:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 Processor -- start 2023-06-23T18:03:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 -- start start 2023-06-23T18:03:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0cc380 0x7fccdc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc160420 0x7fccdc151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc09afb0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc09b440 con 0x7fccdc160420 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce33f9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc09a3a0 con 0x7fccdc0cc380 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47680/0 (socket says 172.21.15.22:47680) 2023-06-23T18:03:51.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 -- 172.21.15.22:0/1255095856 learned_addr learned my addr 172.21.15.22:0/1255095856 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:51.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce2bf8700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc160420 0x7fccdc151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce1bf6700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0cc380 0x7fccdc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 -- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0cc380 msgr2=0x7fccdc0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0cc380 0x7fccdc0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 -- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc160420 msgr2=0x7fccdc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc160420 0x7fccdc151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 -- 172.21.15.22:0/1255095856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0009a70 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.484+0000 7fcce23f7700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccd0014040 tx=0x7fccd0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0012430 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd0020070 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc09a680 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd001ba70 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc09ac40 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc163ec0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fccd001bbd0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 0x7fccc00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:51.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fccd0022070 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.488+0000 7fcce1bf6700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 0x7fccc00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:51.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.492+0000 7fcce1bf6700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 0x7fccc00687a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccdc160ed0 tx=0x7fccd8007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:51.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.500+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fccd0053890 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fccdc0c16a0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fccceffd700 1 -- 172.21.15.22:0/1255095856 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fccd0056eb0 con 0x7fccdc0c0fe0 2023-06-23T18:03:51.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 msgr2=0x7fccc00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 0x7fccc00687a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccdc160ed0 tx=0x7fccd8007e00 comp rx=0 tx=0).stop 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 msgr2=0x7fccdc099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccd0014040 tx=0x7fccd0004af0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 shutdown_connections 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fccc00662f0 0x7fccc00687a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc160420 0x7fccdc151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc0cc380 0x7fccdc0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 --2- 172.21.15.22:0/1255095856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccdc0c0fe0 0x7fccdc099ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 >> 172.21.15.22:0/1255095856 conn(0x7fccdc06a5f0 msgr2=0x7fccdc0acdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.676+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 shutdown_connections 2023-06-23T18:03:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:51.680+0000 7fcce33f9700 1 -- 172.21.15.22:0/1255095856 wait complete. 2023-06-23T18:03:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:03:51.704 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2023-06-23T18:03:51.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:51.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-06-23T18:03:51.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[41389]: fuse finished with error 0 and tester_r 0 2023-06-23T18:03:51.725 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:03:57.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:03:57.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:03:57.828 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-06-23T18:03:57.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:57.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.960+0000 7f0ac3fff700 1 Processor -- start 2023-06-23T18:03:57.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.960+0000 7f0ac3fff700 1 -- start start 2023-06-23T18:03:57.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 0x7f0abc0e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 0x7f0abc039d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0e5260 con 0x7f0abc038980 2023-06-23T18:03:57.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc053500 con 0x7f0abc036320 2023-06-23T18:03:57.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc051950 con 0x7f0abc0514c0 2023-06-23T18:03:57.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac2ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 0x7f0abc0e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac2ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 0x7f0abc0e4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43706/0 (socket says 172.21.15.22:43706) 2023-06-23T18:03:57.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 0x7f0abc039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac2ffd700 1 -- 172.21.15.22:0/1797050249 learned_addr learned my addr 172.21.15.22:0/1797050249 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1797050249 >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 msgr2=0x7f0abc0e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac1ffb700 1 -- 172.21.15.22:0/1797050249 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887531030 0 0) 0x7f0ac40735f0 con 0x7f0abc038980 2023-06-23T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 --2- 172.21.15.22:0/1797050249 >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 0x7f0abc0e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1797050249 >> v1:172.21.15.169:6789/0 conn(0x7f0abc038980 legacy=0x7f0abc055650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1797050249 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc054b70 con 0x7f0abc0514c0 2023-06-23T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac27fc700 1 --2- 172.21.15.22:0/1797050249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 0x7f0abc039d50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009850 tx=0x7f0ab8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340342b2b4139965 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac1ffb700 1 -- 172.21.15.22:0/1797050249 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8011830 con 0x7f0abc0514c0 2023-06-23T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac1ffb700 1 -- 172.21.15.22:0/1797050249 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8011990 con 0x7f0abc0514c0 2023-06-23T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac1ffb700 1 -- 172.21.15.22:0/1797050249 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8011d40 con 0x7f0abc0514c0 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1797050249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 msgr2=0x7f0abc039d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1797050249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 0x7f0abc039d50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009850 tx=0x7f0ab8009b60 comp rx=0 tx=0).stop 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1797050249 shutdown_connections 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1797050249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc0514c0 0x7f0abc039d50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1797050249 >> v2:172.21.15.22:3300/0 conn(0x7f0abc036320 0x7f0abc0e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1797050249 >> 172.21.15.22:0/1797050249 conn(0x7f0abc057b90 msgr2=0x7f0abc053980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1797050249 shutdown_connections 2023-06-23T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.964+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1797050249 wait complete. 2023-06-23T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 Processor -- start 2023-06-23T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 -- start start 2023-06-23T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc036320 0x7f0abc0e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 0x7f0abc0e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0dbfa0 con 0x7f0abc0514c0 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d8670 con 0x7f0abc036320 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d8950 con 0x7f0abc038980 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 0x7f0abc0e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60352/0 (socket says 172.21.15.22:60352) 2023-06-23T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1804437030 learned_addr learned my addr 172.21.15.22:0/1804437030 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:57.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 0x7f0abc0e2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43720/0 (socket says 172.21.15.22:43720) 2023-06-23T18:03:57.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac2ffd700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc036320 0x7f0abc0e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc036320 msgr2=0x7f0abc0e16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc036320 0x7f0abc0e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 msgr2=0x7f0abc0e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 0x7f0abc0e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 -- 172.21.15.22:0/1804437030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80094b0 con 0x7f0abc038980 2023-06-23T18:03:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0ac27fc700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80090b0 tx=0x7f0ab8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8006ea0 con 0x7f0abc038980 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.968+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab801f070 con 0x7f0abc038980 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc03c2c0 con 0x7f0abc038980 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003900 con 0x7f0abc038980 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc0dbcf0 con 0x7f0abc038980 2023-06-23T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc0d84a0 con 0x7f0abc038980 2023-06-23T18:03:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0ab8003a60 con 0x7f0abc038980 2023-06-23T18:03:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0aabfff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 0x7f0ab00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0ac2ffd700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 0x7f0ab00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.972+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ab8021070 con 0x7f0abc038980 2023-06-23T18:03:57.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.976+0000 7f0ac2ffd700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 0x7f0ab00687d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0dfb80 tx=0x7f0ab40092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:57.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:57.984+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0ab8052b00 con 0x7f0abc038980 2023-06-23T18:03:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.148+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0abc0059c0 con 0x7f0abc038980 2023-06-23T18:03:58.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.152+0000 7f0aabfff700 1 -- 172.21.15.22:0/1804437030 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f0ab8056120 con 0x7f0abc038980 2023-06-23T18:03:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:58.168 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 msgr2=0x7f0ab00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 0x7f0ab00687d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0dfb80 tx=0x7f0ab40092e0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 msgr2=0x7f0abc0d7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80090b0 tx=0x7f0ab8004770 comp rx=0 tx=0).stop 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 shutdown_connections 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0abc0514c0 0x7f0abc0e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0abc038980 0x7f0abc0d7f60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ab0066320 0x7f0ab00687d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 --2- 172.21.15.22:0/1804437030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0abc036320 0x7f0abc0e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 >> 172.21.15.22:0/1804437030 conn(0x7f0abc057b90 msgr2=0x7f0abc053910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:58.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 shutdown_connections 2023-06-23T18:03:58.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.156+0000 7f0ac3fff700 1 -- 172.21.15.22:0/1804437030 wait complete. 2023-06-23T18:03:58.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:03:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 Processor -- start 2023-06-23T18:03:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 -- start start 2023-06-23T18:03:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f732415c280 0x7f73240c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f732406d110 0x7f732415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73240ca370 con 0x7f732415beb0 2023-06-23T18:03:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240c8790 con 0x7f732406d110 2023-06-23T18:03:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7329332700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7324160780 con 0x7f732415c280 2023-06-23T18:03:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7328b31700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f732406d110 0x7f732415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7328b31700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f732406d110 0x7f732415fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43724/0 (socket says 172.21.15.22:43724) 2023-06-23T18:03:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f7328b31700 1 -- 172.21.15.22:0/2317928274 learned_addr learned my addr 172.21.15.22:0/2317928274 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.288+0000 7f73237fe700 1 --2- 172.21.15.22:0/2317928274 >> v2:172.21.15.169:3300/0 conn(0x7f732415c280 0x7f73240c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7328b31700 1 -- 172.21.15.22:0/2317928274 >> v2:172.21.15.169:3300/0 conn(0x7f732415c280 msgr2=0x7f73240c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:58.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7328b31700 1 --2- 172.21.15.22:0/2317928274 >> v2:172.21.15.169:3300/0 conn(0x7f732415c280 0x7f73240c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7328b31700 1 -- 172.21.15.22:0/2317928274 >> v1:172.21.15.169:6789/0 conn(0x7f732415beb0 legacy=0x7f7324070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7328b31700 1 -- 172.21.15.22:0/2317928274 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324071920 con 0x7f732406d110 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7328b31700 1 --2- 172.21.15.22:0/2317928274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f732415fc00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7324071b10 tx=0x7f731c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33da257d9e6ba7c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7322ffd700 1 -- 172.21.15.22:0/2317928274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c004230 con 0x7f732406d110 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7322ffd700 1 -- 172.21.15.22:0/2317928274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c004390 con 0x7f732406d110 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7322ffd700 1 -- 172.21.15.22:0/2317928274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0036c0 con 0x7f732406d110 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- 172.21.15.22:0/2317928274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 msgr2=0x7f732415fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 --2- 172.21.15.22:0/2317928274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f732415fc00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7324071b10 tx=0x7f731c009bf0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- 172.21.15.22:0/2317928274 shutdown_connections 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 --2- 172.21.15.22:0/2317928274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f732415fc00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 --2- 172.21.15.22:0/2317928274 >> v2:172.21.15.169:3300/0 conn(0x7f732415c280 0x7f73240c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- 172.21.15.22:0/2317928274 >> 172.21.15.22:0/2317928274 conn(0x7f732406a5f0 msgr2=0x7f73240caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- 172.21.15.22:0/2317928274 shutdown_connections 2023-06-23T18:03:58.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- 172.21.15.22:0/2317928274 wait complete. 2023-06-23T18:03:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 Processor -- start 2023-06-23T18:03:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.292+0000 7f7329332700 1 -- start start 2023-06-23T18:03:58.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 0x7f73240b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f732415c280 0x7f73241586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c6540 con 0x7f732406d110 2023-06-23T18:03:58.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732415d410 con 0x7f732415beb0 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c5620 con 0x7f732415c280 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43740/0 (socket says 172.21.15.22:43740) 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 -- 172.21.15.22:0/2220350555 learned_addr learned my addr 172.21.15.22:0/2220350555 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7328b31700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f732415c280 0x7f73241586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f73237fe700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 0x7f73240b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 -- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f732415c280 msgr2=0x7f73241586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f732415c280 0x7f73241586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 -- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 msgr2=0x7f73240b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 0x7f73240b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 -- 172.21.15.22:0/2220350555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c009570 con 0x7f732406d110 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f73237fe700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 0x7f73240b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:03:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7323fff700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f731800c500 tx=0x7f731800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180052d0 con 0x7f732406d110 2023-06-23T18:03:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318005430 con 0x7f732406d110 2023-06-23T18:03:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73240c58c0 con 0x7f732406d110 2023-06-23T18:03:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180055a0 con 0x7f732406d110 2023-06-23T18:03:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.296+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73240c5e80 con 0x7f732406d110 2023-06-23T18:03:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73240c6680 con 0x7f732406d110 2023-06-23T18:03:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f73180036a0 con 0x7f732406d110 2023-06-23T18:03:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f7320ff9700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 0x7f73100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7318083260 con 0x7f732406d110 2023-06-23T18:03:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f73237fe700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 0x7f73100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.300+0000 7f73237fe700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 0x7f73100687a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f73240af110 tx=0x7f731400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.304+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7318050480 con 0x7f732406d110 2023-06-23T18:03:58.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.464+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f732404e4c0 con 0x7f732406d110 2023-06-23T18:03:58.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.468+0000 7f7320ff9700 1 -- 172.21.15.22:0/2220350555 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f7318053aa0 con 0x7f732406d110 2023-06-23T18:03:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:03:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 msgr2=0x7f73100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 0x7f73100687a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f73240af110 tx=0x7f731400b040 comp rx=0 tx=0).stop 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 msgr2=0x7f73240c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f731800c500 tx=0x7f731800c8c0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 shutdown_connections 2023-06-23T18:03:58.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f732415c280 0x7f73241586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f73100662f0 0x7f73100687a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f732415beb0 0x7f73240b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 --2- 172.21.15.22:0/2220350555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f732406d110 0x7f73240c50e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 >> 172.21.15.22:0/2220350555 conn(0x7f732406a5f0 msgr2=0x7f73240cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 shutdown_connections 2023-06-23T18:03:58.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.472+0000 7f7329332700 1 -- 172.21.15.22:0/2220350555 wait complete. 2023-06-23T18:03:58.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T18:03:58.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.604+0000 7ff294897700 1 Processor -- start 2023-06-23T18:03:58.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.604+0000 7ff294897700 1 -- start start 2023-06-23T18:03:58.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff290152ea0 0x7ff290160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2900ce7e0 0x7ff29006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2900cbc50 con 0x7ff29006df70 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29015c470 con 0x7ff290152ea0 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29006e3f0 con 0x7ff2900ce7e0 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff290152ea0 0x7ff290160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28ed9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff29006df70 0x7ff290069960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37072/0 (socket says 172.21.15.22:37072) 2023-06-23T18:03:58.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28ed9d700 1 -- 172.21.15.22:0/982440515 learned_addr learned my addr 172.21.15.22:0/982440515 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:58.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28dd9b700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2900ce7e0 0x7ff29006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 -- 172.21.15.22:0/982440515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2900ce7e0 msgr2=0x7ff29006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:03:58.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2900ce7e0 0x7ff29006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 -- 172.21.15.22:0/982440515 >> v1:172.21.15.22:6789/0 conn(0x7ff29006df70 legacy=0x7ff290069960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28d59a700 1 -- 172.21.15.22:0/982440515 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712162238 0 0) 0x7ff2900cbc50 con 0x7ff29006df70 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 -- 172.21.15.22:0/982440515 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290071de0 con 0x7ff290152ea0 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28e59c700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290160530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff29006e530 tx=0x7ff280009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a418924401f5c92b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28d59a700 1 -- 172.21.15.22:0/982440515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280004020 con 0x7ff290152ea0 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28d59a700 1 -- 172.21.15.22:0/982440515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280004180 con 0x7ff290152ea0 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff28d59a700 1 -- 172.21.15.22:0/982440515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280004530 con 0x7ff290152ea0 2023-06-23T18:03:58.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- 172.21.15.22:0/982440515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 msgr2=0x7ff290160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290160530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff29006e530 tx=0x7ff280009b30 comp rx=0 tx=0).stop 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- 172.21.15.22:0/982440515 shutdown_connections 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2900ce7e0 0x7ff29006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 --2- 172.21.15.22:0/982440515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290160530 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- 172.21.15.22:0/982440515 >> 172.21.15.22:0/982440515 conn(0x7ff29006e980 msgr2=0x7ff29006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- 172.21.15.22:0/982440515 shutdown_connections 2023-06-23T18:03:58.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.608+0000 7ff294897700 1 -- 172.21.15.22:0/982440515 wait complete. 2023-06-23T18:03:58.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.612+0000 7ff294897700 1 Processor -- start 2023-06-23T18:03:58.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.612+0000 7ff294897700 1 -- start start 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff29006df70 0x7ff290154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 0x7ff290140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290154920 con 0x7ff290152ea0 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290153d10 con 0x7ff2900ce7e0 2023-06-23T18:03:58.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290153ff0 con 0x7ff29006df70 2023-06-23T18:03:58.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43768/0 (socket says 172.21.15.22:43768) 2023-06-23T18:03:58.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 -- 172.21.15.22:0/70446745 learned_addr learned my addr 172.21.15.22:0/70446745 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:58.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28e59c700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff29006df70 0x7ff290154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28dd9b700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 0x7ff290140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 -- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff29006df70 msgr2=0x7ff290154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff29006df70 0x7ff290154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 -- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 msgr2=0x7ff290140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 0x7ff290140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 -- 172.21.15.22:0/70446745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800094b0 con 0x7ff290152ea0 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28dd9b700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 0x7ff290140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff28ed9d700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff288002230 tx=0x7ff28800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800ab70 con 0x7ff290152ea0 2023-06-23T18:03:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28800acf0 con 0x7ff290152ea0 2023-06-23T18:03:58.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2901542d0 con 0x7ff290152ea0 2023-06-23T18:03:58.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288007310 con 0x7ff290152ea0 2023-06-23T18:03:58.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.616+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff290141130 con 0x7ff290152ea0 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff290060eb0 con 0x7ff290152ea0 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff288007470 con 0x7ff290152ea0 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff286ffd700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 0x7ff2700687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff288083de0 con 0x7ff290152ea0 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff28e59c700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 0x7ff2700687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:58.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.620+0000 7ff28e59c700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 0x7ff2700687d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2800090b0 tx=0x7ff280008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:58.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.628+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff288050f70 con 0x7ff290152ea0 2023-06-23T18:03:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:58.832+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff29004dc70 con 0x7ff290152ea0 2023-06-23T18:03:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.692+0000 7ff286ffd700 1 -- 172.21.15.22:0/70446745 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff2880180b0 con 0x7ff290152ea0 2023-06-23T18:03:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 msgr2=0x7ff2700687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 0x7ff2700687d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2800090b0 tx=0x7ff280008030 comp rx=0 tx=0).stop 2023-06-23T18:03:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 msgr2=0x7ff290153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff288002230 tx=0x7ff28800cf50 comp rx=0 tx=0).stop 2023-06-23T18:03:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 shutdown_connections 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff290152ea0 0x7ff290153270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff270066320 0x7ff2700687d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ce7e0 0x7ff290140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 --2- 172.21.15.22:0/70446745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff29006df70 0x7ff290154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 >> 172.21.15.22:0/70446745 conn(0x7ff29006e980 msgr2=0x7ff29014fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 shutdown_connections 2023-06-23T18:03:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.696+0000 7ff294897700 1 -- 172.21.15.22:0/70446745 wait complete. 2023-06-23T18:03:59.729 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-06-23T18:03:59.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T18:03:59.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 Processor -- start 2023-06-23T18:03:59.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 -- start start 2023-06-23T18:03:59.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f51a40b5ad0 0x7f51a40d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:59.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a40d0420 con 0x7f51a40b4730 2023-06-23T18:03:59.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a40cd670 con 0x7f51a415f3b0 2023-06-23T18:03:59.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a970d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40cd950 con 0x7f51a40b5ad0 2023-06-23T18:03:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a27fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f51a40b4730 0x7f51a40b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37080/0 (socket says 172.21.15.22:37080) 2023-06-23T18:03:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a27fc700 1 -- 172.21.15.22:0/3372795954 learned_addr learned my addr 172.21.15.22:0/3372795954 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:03:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.844+0000 7f51a37fe700 1 --2- 172.21.15.22:0/3372795954 >> v2:172.21.15.169:3300/0 conn(0x7f51a40b5ad0 0x7f51a40d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a1ffb700 1 -- 172.21.15.22:0/3372795954 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367714407 0 0) 0x7f51a40d0420 con 0x7f51a40b4730 2023-06-23T18:03:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a1ffb700 1 -- 172.21.15.22:0/3372795954 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51900035d0 con 0x7f51a40b4730 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a37fe700 1 -- 172.21.15.22:0/3372795954 >> v1:172.21.15.169:6789/0 conn(0x7f51a415f3b0 legacy=0x7f51a40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a37fe700 1 -- 172.21.15.22:0/3372795954 >> v1:172.21.15.22:6789/0 conn(0x7f51a40b4730 legacy=0x7f51a40b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a37fe700 1 -- 172.21.15.22:0/3372795954 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40cdc30 con 0x7f51a40b5ad0 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a37fe700 1 --2- 172.21.15.22:0/3372795954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a40b5ad0 0x7f51a40d1cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f519c009d90 tx=0x7f519c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c8af15e79d96a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a1ffb700 1 -- 172.21.15.22:0/3372795954 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c00e070 con 0x7f51a40b5ad0 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a1ffb700 1 -- 172.21.15.22:0/3372795954 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c00bb20 con 0x7f51a40b5ad0 2023-06-23T18:03:59.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a1ffb700 1 -- 172.21.15.22:0/3372795954 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c012430 con 0x7f51a40b5ad0 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 -- 172.21.15.22:0/3372795954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a40b5ad0 msgr2=0x7f51a40d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 --2- 172.21.15.22:0/3372795954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a40b5ad0 0x7f51a40d1cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f519c009d90 tx=0x7f519c00b330 comp rx=0 tx=0).stop 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 -- 172.21.15.22:0/3372795954 shutdown_connections 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 --2- 172.21.15.22:0/3372795954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a40b5ad0 0x7f51a40d1cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 -- 172.21.15.22:0/3372795954 >> 172.21.15.22:0/3372795954 conn(0x7f51a4069960 msgr2=0x7f51a4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 -- 172.21.15.22:0/3372795954 shutdown_connections 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.848+0000 7f51a970d700 1 -- 172.21.15.22:0/3372795954 wait complete. 2023-06-23T18:03:59.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 Processor -- start 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- start start 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51a40b5ad0 0x7f51a40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a415f3b0 0x7f51a40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40c60a0 con 0x7f51a40b4730 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41f4db0 con 0x7f51a415f3b0 2023-06-23T18:03:59.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41f5090 con 0x7f51a40b5ad0 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43782/0 (socket says 172.21.15.22:43782) 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 -- 172.21.15.22:0/2118798719 learned_addr learned my addr 172.21.15.22:0/2118798719 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a37fe700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a415f3b0 0x7f51a40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a27fc700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51a40b5ad0 0x7f51a40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 -- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51a40b5ad0 msgr2=0x7f51a40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51a40b5ad0 0x7f51a40b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 -- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a415f3b0 msgr2=0x7f51a40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a415f3b0 0x7f51a40cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 -- 172.21.15.22:0/2118798719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c009a70 con 0x7f51a40b4730 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a2ffd700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c500 tx=0x7f518c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0052d0 con 0x7f51a40b4730 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c005430 con 0x7f51a40b4730 2023-06-23T18:03:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a41f5370 con 0x7f51a40b4730 2023-06-23T18:03:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0055a0 con 0x7f51a40b4730 2023-06-23T18:03:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.852+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a41f17e0 con 0x7f51a40b4730 2023-06-23T18:03:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.856+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a40c6210 con 0x7f51a40b4730 2023-06-23T18:03:59.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.856+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f518c0038e0 con 0x7f51a40b4730 2023-06-23T18:03:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.856+0000 7f519b7fe700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 0x7f51840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:03:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.856+0000 7f51a27fc700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 0x7f51840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:03:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.856+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f518c0833c0 con 0x7f51a40b4730 2023-06-23T18:03:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.860+0000 7f51a27fc700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 0x7f51840687a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51a415c420 tx=0x7f5194007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:03:59.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:03:59.864+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f51a40c6210 con 0x7f51a40b4730 2023-06-23T18:04:00.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.064+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51a40c61e0 con 0x7f51a40b4730 2023-06-23T18:04:00.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.703+0000 7f519b7fe700 1 -- 172.21.15.22:0/2118798719 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f518c0504b0 con 0x7f51a40b4730 2023-06-23T18:04:00.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 msgr2=0x7f51840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 0x7f51840687a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51a415c420 tx=0x7f5194007de0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 msgr2=0x7f51a41f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c500 tx=0x7f518c00c810 comp rx=0 tx=0).stop 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 shutdown_connections 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f51840662f0 0x7f51840687a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a415f3b0 0x7f51a40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51a40b5ad0 0x7f51a40b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 --2- 172.21.15.22:0/2118798719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a40b4730 0x7f51a41f12a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 >> 172.21.15.22:0/2118798719 conn(0x7f51a4069960 msgr2=0x7f51a40b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:00.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 shutdown_connections 2023-06-23T18:04:00.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.707+0000 7f51a970d700 1 -- 172.21.15.22:0/2118798719 wait complete. 2023-06-23T18:04:00.740 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-23T18:04:00.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:00.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.843+0000 7f476b59e700 1 Processor -- start 2023-06-23T18:04:00.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.843+0000 7f476b59e700 1 -- start start 2023-06-23T18:04:00.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f476c0c0fe0 0x7f476c0c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0c18f0 con 0x7f476c160420 2023-06-23T18:04:00.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c163ec0 con 0x7f476c0c0fe0 2023-06-23T18:04:00.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1641a0 con 0x7f476c0cc380 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4761d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f476c160420 0x7f476c0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37094/0 (socket says 172.21.15.22:37094) 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4761d9b700 1 -- 172.21.15.22:0/3210543363 learned_addr learned my addr 172.21.15.22:0/3210543363 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 --2- 172.21.15.22:0/3210543363 >> v2:172.21.15.22:3300/0 conn(0x7f476c0c0fe0 0x7f476c0c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476ad9d700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 -- 172.21.15.22:0/3210543363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 msgr2=0x7f476c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:00.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c0c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 -- 172.21.15.22:0/3210543363 >> v1:172.21.15.22:6789/0 conn(0x7f476c160420 legacy=0x7f476c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4769d9b700 1 -- 172.21.15.22:0/3210543363 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498336949 0 0) 0x7f476c0c18f0 con 0x7f476c160420 2023-06-23T18:04:00.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 -- 172.21.15.22:0/3210543363 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c164480 con 0x7f476c0c0fe0 2023-06-23T18:04:00.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476a59c700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0c13b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4754009850 tx=0x7f4754009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1a5b041033b876 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:00.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4769d9b700 1 -- 172.21.15.22:0/3210543363 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754011830 con 0x7f476c0c0fe0 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4769d9b700 1 -- 172.21.15.22:0/3210543363 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754011990 con 0x7f476c0c0fe0 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f4769d9b700 1 -- 172.21.15.22:0/3210543363 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754011d40 con 0x7f476c0c0fe0 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- 172.21.15.22:0/3210543363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 msgr2=0x7f476c0c13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0c13b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4754009850 tx=0x7f4754009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- 172.21.15.22:0/3210543363 shutdown_connections 2023-06-23T18:04:00.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 --2- 172.21.15.22:0/3210543363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0c13b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- 172.21.15.22:0/3210543363 >> 172.21.15.22:0/3210543363 conn(0x7f476c06a5f0 msgr2=0x7f476c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:00.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- 172.21.15.22:0/3210543363 shutdown_connections 2023-06-23T18:04:00.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.847+0000 7f476b59e700 1 -- 172.21.15.22:0/3210543363 wait complete. 2023-06-23T18:04:00.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.851+0000 7f476b59e700 1 Processor -- start 2023-06-23T18:04:00.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.851+0000 7f476b59e700 1 -- start start 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c160420 0x7f476c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c152c30 con 0x7f476c0c0fe0 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c14c3e0 con 0x7f476c160420 2023-06-23T18:04:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0b0e40 con 0x7f476c0cc380 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43792/0 (socket says 172.21.15.22:43792) 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 -- 172.21.15.22:0/1523703813 learned_addr learned my addr 172.21.15.22:0/1523703813 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f4761d9b700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476ad9d700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c160420 0x7f476c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 -- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 msgr2=0x7f476c1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 -- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c160420 msgr2=0x7f476c0c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:00.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c160420 0x7f476c0c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:00.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 -- 172.21.15.22:0/1523703813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540094b0 con 0x7f476c0c0fe0 2023-06-23T18:04:00.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476a59c700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f476c06dd50 tx=0x7f4754004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475401aac0 con 0x7f476c0c0fe0 2023-06-23T18:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475401f070 con 0x7f476c0c0fe0 2023-06-23T18:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0b3ca0 con 0x7f476c0c0fe0 2023-06-23T18:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475401ae60 con 0x7f476c0c0fe0 2023-06-23T18:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c0b1390 con 0x7f476c0c0fe0 2023-06-23T18:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.855+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c04e4c0 con 0x7f476c0c0fe0 2023-06-23T18:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.859+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f475401b430 con 0x7f476c0c0fe0 2023-06-23T18:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.859+0000 7f47637fe700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 0x7f474c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.859+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4754085a00 con 0x7f476c0c0fe0 2023-06-23T18:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.859+0000 7f4761d9b700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 0x7f474c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:00.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.859+0000 7f4761d9b700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 0x7f474c0687a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ad1c0 tx=0x7f475c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:00.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:00.867+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4754052c00 con 0x7f476c0c0fe0 2023-06-23T18:04:01.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.063+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f476c0c9020 con 0x7f476c0c0fe0 2023-06-23T18:04:01.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f47637fe700 1 -- 172.21.15.22:0/1523703813 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f4754015090 con 0x7f476c0c0fe0 2023-06-23T18:04:01.081 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:01.081 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 msgr2=0x7f474c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 0x7f474c0687a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ad1c0 tx=0x7f475c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 msgr2=0x7f476c0b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f476c06dd50 tx=0x7f4754004770 comp rx=0 tx=0).stop 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 shutdown_connections 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c0662f0 0x7f474c0687a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c160420 0x7f476c0c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0cc380 0x7f476c1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 --2- 172.21.15.22:0/1523703813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0c0fe0 0x7f476c0b3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 >> 172.21.15.22:0/1523703813 conn(0x7f476c06a5f0 msgr2=0x7f476c070480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 shutdown_connections 2023-06-23T18:04:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.067+0000 7f476b59e700 1 -- 172.21.15.22:0/1523703813 wait complete. 2023-06-23T18:04:01.085 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:01.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.191+0000 7f861a3b7700 1 Processor -- start 2023-06-23T18:04:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.191+0000 7f861a3b7700 1 -- start start 2023-06-23T18:04:01.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 0x7f861415de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86140c9d30 0x7f861415e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86140ceff0 con 0x7f86140cb620 2023-06-23T18:04:01.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861415d990 con 0x7f86140c9d30 2023-06-23T18:04:01.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86140cb9f0 con 0x7f86140ceb60 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8613fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f86140cb620 0x7f86140ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37096/0 (socket says 172.21.15.22:37096) 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8613fff700 1 -- 172.21.15.22:0/1428125001 learned_addr learned my addr 172.21.15.22:0/1428125001 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 --2- 172.21.15.22:0/1428125001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 0x7f861415de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8618954700 1 --2- 172.21.15.22:0/1428125001 >> v2:172.21.15.169:3300/0 conn(0x7f86140c9d30 0x7f861415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8612ffd700 1 -- 172.21.15.22:0/1428125001 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189816575 0 0) 0x7f86140ceff0 con 0x7f86140cb620 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8612ffd700 1 -- 172.21.15.22:0/1428125001 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85fc0035d0 con 0x7f86140cb620 2023-06-23T18:04:01.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 -- 172.21.15.22:0/1428125001 >> v2:172.21.15.169:3300/0 conn(0x7f86140c9d30 msgr2=0x7f861415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:01.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 --2- 172.21.15.22:0/1428125001 >> v2:172.21.15.169:3300/0 conn(0x7f86140c9d30 0x7f861415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 -- 172.21.15.22:0/1428125001 >> v1:172.21.15.22:6789/0 conn(0x7f86140cb620 legacy=0x7f86140ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 -- 172.21.15.22:0/1428125001 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861406c670 con 0x7f86140ceb60 2023-06-23T18:04:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f86137fe700 1 --2- 172.21.15.22:0/1428125001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 0x7f861415de20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8604009d90 tx=0x7f860400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95653a0d7bfa1530 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8612ffd700 1 -- 172.21.15.22:0/1428125001 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400e070 con 0x7f86140ceb60 2023-06-23T18:04:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8612ffd700 1 -- 172.21.15.22:0/1428125001 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400bb20 con 0x7f86140ceb60 2023-06-23T18:04:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f8612ffd700 1 -- 172.21.15.22:0/1428125001 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604012430 con 0x7f86140ceb60 2023-06-23T18:04:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- 172.21.15.22:0/1428125001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 msgr2=0x7f861415de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 --2- 172.21.15.22:0/1428125001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 0x7f861415de20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8604009d90 tx=0x7f860400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- 172.21.15.22:0/1428125001 shutdown_connections 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 --2- 172.21.15.22:0/1428125001 >> v2:172.21.15.169:3300/0 conn(0x7f86140c9d30 0x7f861415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 --2- 172.21.15.22:0/1428125001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140ceb60 0x7f861415de20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- 172.21.15.22:0/1428125001 >> 172.21.15.22:0/1428125001 conn(0x7f8614069960 msgr2=0x7f8614069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- 172.21.15.22:0/1428125001 shutdown_connections 2023-06-23T18:04:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.195+0000 7f861a3b7700 1 -- 172.21.15.22:0/1428125001 wait complete. 2023-06-23T18:04:01.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 Processor -- start 2023-06-23T18:04:01.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- start start 2023-06-23T18:04:01.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140c9d30 0x7f86140af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86140cb620 0x7f86140b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861415a6b0 con 0x7f86140ceb60 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614150b60 con 0x7f86140cb620 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86141505a0 con 0x7f86140c9d30 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43800/0 (socket says 172.21.15.22:43800) 2023-06-23T18:04:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 -- 172.21.15.22:0/2681286201 learned_addr learned my addr 172.21.15.22:0/2681286201 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f86137fe700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86140cb620 0x7f86140b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8613fff700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140c9d30 0x7f86140af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 -- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140c9d30 msgr2=0x7f86140af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140c9d30 0x7f86140af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 -- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86140cb620 msgr2=0x7f86140b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86140cb620 0x7f86140b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 -- 172.21.15.22:0/2681286201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009a70 con 0x7f86140ceb60 2023-06-23T18:04:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8618954700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f860c00e7f0 tx=0x7f860c00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c0095c0 con 0x7f86140ceb60 2023-06-23T18:04:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c013680 con 0x7f86140ceb60 2023-06-23T18:04:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8614150880 con 0x7f86140ceb60 2023-06-23T18:04:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c013a10 con 0x7f86140ceb60 2023-06-23T18:04:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.199+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8614150230 con 0x7f86140ceb60 2023-06-23T18:04:01.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.203+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861415a820 con 0x7f86140ceb60 2023-06-23T18:04:01.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.203+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f860c013b70 con 0x7f86140ceb60 2023-06-23T18:04:01.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.203+0000 7f8610ff9700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 0x7f86000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.203+0000 7f8613fff700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 0x7f86000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.203+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f860c083f50 con 0x7f86140ceb60 2023-06-23T18:04:01.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.207+0000 7f8613fff700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 0x7f86000687a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86080011e0 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.211+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f860c0510d0 con 0x7f86140ceb60 2023-06-23T18:04:01.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.387+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f861404dc70 con 0x7f86140ceb60 2023-06-23T18:04:01.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.387+0000 7f8610ff9700 1 -- 172.21.15.22:0/2681286201 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f860c0546f0 con 0x7f86140ceb60 2023-06-23T18:04:01.403 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:01.404 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:01.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 msgr2=0x7f86000687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 0x7f86000687a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86080011e0 tx=0x7f8608009040 comp rx=0 tx=0).stop 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 msgr2=0x7f86141e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f860c00e7f0 tx=0x7f860c00eb00 comp rx=0 tx=0).stop 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 shutdown_connections 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86140ceb60 0x7f86141e3cb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86000662f0 0x7f86000687a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86140cb620 0x7f86140b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 --2- 172.21.15.22:0/2681286201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86140c9d30 0x7f86140af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 >> 172.21.15.22:0/2681286201 conn(0x7f8614069960 msgr2=0x7f86141527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 shutdown_connections 2023-06-23T18:04:01.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.391+0000 7f861a3b7700 1 -- 172.21.15.22:0/2681286201 wait complete. 2023-06-23T18:04:01.419 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-23T18:04:01.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:04:01.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 Processor -- start 2023-06-23T18:04:01.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 -- start start 2023-06-23T18:04:01.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c700dd280 0x7f9c70050760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c70052bf0 con 0x7f9c700dfb70 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c700dd650 con 0x7f9c700dd280 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c7cdc6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700528c0 con 0x7f9c70036e80 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c700dd280 0x7f9c70050760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c700dd280 0x7f9c70050760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:42680/0 (socket says 172.21.15.22:42680) 2023-06-23T18:04:01.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60394/0 (socket says 172.21.15.22:60394) 2023-06-23T18:04:01.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 -- 172.21.15.22:0/3726888756 learned_addr learned my addr 172.21.15.22:0/3726888756 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:01.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c76ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9c700dfb70 0x7f9c70036770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59810/0 (socket says 172.21.15.22:59810) 2023-06-23T18:04:01.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 -- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 msgr2=0x7f9c70053080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:01.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 -- 172.21.15.22:0/3726888756 >> v1:172.21.15.169:6789/0 conn(0x7f9c700dfb70 legacy=0x7f9c70036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c777fe700 1 -- 172.21.15.22:0/3726888756 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c700e0430 con 0x7f9c700dd280 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.515+0000 7f9c77fff700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c777fe700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dd280 0x7f9c70050760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009910 tx=0x7f9c6c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad8cda5d3e7d196 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c767fc700 1 -- 172.21.15.22:0/3726888756 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00b850 con 0x7f9c700dd280 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c767fc700 1 -- 172.21.15.22:0/3726888756 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00b9b0 con 0x7f9c700dd280 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c767fc700 1 -- 172.21.15.22:0/3726888756 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00bd40 con 0x7f9c700dd280 2023-06-23T18:04:01.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dd280 msgr2=0x7f9c70050760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dd280 0x7f9c70050760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009910 tx=0x7f9c6c009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/3726888756 shutdown_connections 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c70053080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/3726888756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dd280 0x7f9c70050760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/3726888756 >> 172.21.15.22:0/3726888756 conn(0x7f9c70051770 msgr2=0x7f9c70051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/3726888756 shutdown_connections 2023-06-23T18:04:01.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/3726888756 wait complete. 2023-06-23T18:04:01.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 Processor -- start 2023-06-23T18:04:01.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.519+0000 7f9c7cdc6700 1 -- start start 2023-06-23T18:04:01.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c7004b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dfb70 0x7f9c700d3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700061b0 con 0x7f9c700dd280 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700063e0 con 0x7f9c700dfb70 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700066c0 con 0x7f9c70036e80 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c777fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c7004b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43806/0 (socket says 172.21.15.22:43806) 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 -- 172.21.15.22:0/2041758805 learned_addr learned my addr 172.21.15.22:0/2041758805 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c77fff700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dfb70 0x7f9c700d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 -- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 msgr2=0x7f9c7004b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c7004b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 -- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dfb70 msgr2=0x7f9c700d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dfb70 0x7f9c700d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 -- 172.21.15.22:0/2041758805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009570 con 0x7f9c700dd280 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c777fe700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c7004b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c76ffd700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c700 tx=0x7f9c6400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64005cd0 con 0x7f9c700dd280 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c64005e30 con 0x7f9c700dd280 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c700069a0 con 0x7f9c700dd280 2023-06-23T18:04:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640072d0 con 0x7f9c700dd280 2023-06-23T18:04:01.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c70006f60 con 0x7f9c700dd280 2023-06-23T18:04:01.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.523+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c70007150 con 0x7f9c700dd280 2023-06-23T18:04:01.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.527+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9c64007430 con 0x7f9c700dd280 2023-06-23T18:04:01.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.527+0000 7f9c63fff700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 0x7f9c5c0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.527+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9c640832b0 con 0x7f9c700dd280 2023-06-23T18:04:01.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.527+0000 7f9c777fe700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 0x7f9c5c0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.527+0000 7f9c777fe700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 0x7f9c5c0687f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c000c00 tx=0x7f9c6c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.531+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f9c64050560 con 0x7f9c700dd280 2023-06-23T18:04:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.695+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c700e09f0 con 0x7f9c5c066340 2023-06-23T18:04:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.715+0000 7f9c63fff700 1 -- 172.21.15.22:0/2041758805 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367266 (secure 0 0 0) 0x7f9c6c00bcd0 con 0x7f9c5c066340 2023-06-23T18:04:01.734 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.719+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 msgr2=0x7f9c5c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.719+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 0x7f9c5c0687f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c000c00 tx=0x7f9c6c008030 comp rx=0 tx=0).stop 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.719+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 msgr2=0x7f9c70005c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.719+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c700 tx=0x7f9c6400ca10 comp rx=0 tx=0).stop 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 shutdown_connections 2023-06-23T18:04:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9c5c066340 0x7f9c5c0687f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c700dfb70 0x7f9c700d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c700dd280 0x7f9c70005c70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 --2- 172.21.15.22:0/2041758805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c70036e80 0x7f9c7004b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 >> 172.21.15.22:0/2041758805 conn(0x7f9c70051770 msgr2=0x7f9c7004fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 shutdown_connections 2023-06-23T18:04:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.723+0000 7f9c7cdc6700 1 -- 172.21.15.22:0/2041758805 wait complete. 2023-06-23T18:04:01.739 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:04:01.751 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-06-23T18:04:01.615485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235208,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":3284992,"data_stored":2027694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004868"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915552+0000","last_change":"2023-06-23T18:03:41.789565+0000","last_active":"2023-06-23T18:03:42.915552+0000","last_peered":"2023-06-23T18:03:42.915552+0000","last_clean":"2023-06-23T18:03:42.915552+0000","last_became_active":"2023-06-23T18:03:41.787653+0000","last_became_peered":"2023-06-23T18:03:41.787653+0000","last_unstale":"2023-06-23T18:03:42.915552+0000","last_undegraded":"2023-06-23T18:03:42.915552+0000","last_fullsized":"2023-06-23T18:03:42.915552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922374+0000","last_change":"2023-06-23T18:03:37.377461+0000","last_active":"2023-06-23T18:03:42.922374+0000","last_peered":"2023-06-23T18:03:42.922374+0000","last_clean":"2023-06-23T18:03:42.922374+0000","last_became_active":"2023-06-23T18:03:37.377017+0000","last_became_peered":"2023-06-23T18:03:37.377017+0000","last_unstale":"2023-06-23T18:03:42.922374+0000","last_undegraded":"2023-06-23T18:03:42.922374+0000","last_fullsized":"2023-06-23T18:03:42.922374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157012+0000","last_change":"2023-06-23T18:03:37.756400+0000","last_active":"2023-06-23T18:03:42.157012+0000","last_peered":"2023-06-23T18:03:42.157012+0000","last_clean":"2023-06-23T18:03:42.157012+0000","last_became_active":"2023-06-23T18:03:37.755870+0000","last_became_peered":"2023-06-23T18:03:37.755870+0000","last_unstale":"2023-06-23T18:03:42.157012+0000","last_undegraded":"2023-06-23T18:03:42.157012+0000","last_fullsized":"2023-06-23T18:03:42.157012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:35:48.712508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157286+0000","last_change":"2023-06-23T18:03:41.784738+0000","last_active":"2023-06-23T18:03:42.157286+0000","last_peered":"2023-06-23T18:03:42.157286+0000","last_clean":"2023-06-23T18:03:42.157286+0000","last_became_active":"2023-06-23T18:03:41.784348+0000","last_became_peered":"2023-06-23T18:03:41.784348+0000","last_unstale":"2023-06-23T18:03:42.157286+0000","last_undegraded":"2023-06-23T18:03:42.157286+0000","last_fullsized":"2023-06-23T18:03:42.157286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:06:40.211756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920797+0000","last_change":"2023-06-23T18:03:37.364428+0000","last_active":"2023-06-23T18:03:42.920797+0000","last_peered":"2023-06-23T18:03:42.920797+0000","last_clean":"2023-06-23T18:03:42.920797+0000","last_became_active":"2023-06-23T18:03:37.364114+0000","last_became_peered":"2023-06-23T18:03:37.364114+0000","last_unstale":"2023-06-23T18:03:42.920797+0000","last_undegraded":"2023-06-23T18:03:42.920797+0000","last_fullsized":"2023-06-23T18:03:42.920797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:53.340210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917007+0000","last_change":"2023-06-23T18:03:37.761252+0000","last_active":"2023-06-23T18:03:42.917007+0000","last_peered":"2023-06-23T18:03:42.917007+0000","last_clean":"2023-06-23T18:03:42.917007+0000","last_became_active":"2023-06-23T18:03:37.760991+0000","last_became_peered":"2023-06-23T18:03:37.760991+0000","last_unstale":"2023-06-23T18:03:42.917007+0000","last_undegraded":"2023-06-23T18:03:42.917007+0000","last_fullsized":"2023-06-23T18:03:42.917007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:26.556145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922896+0000","last_change":"2023-06-23T18:03:41.789561+0000","last_active":"2023-06-23T18:03:42.922896+0000","last_peered":"2023-06-23T18:03:42.922896+0000","last_clean":"2023-06-23T18:03:42.922896+0000","last_became_active":"2023-06-23T18:03:41.788768+0000","last_became_peered":"2023-06-23T18:03:41.788768+0000","last_unstale":"2023-06-23T18:03:42.922896+0000","last_undegraded":"2023-06-23T18:03:42.922896+0000","last_fullsized":"2023-06-23T18:03:42.922896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924155+0000","last_change":"2023-06-23T18:03:37.377124+0000","last_active":"2023-06-23T18:03:42.924155+0000","last_peered":"2023-06-23T18:03:42.924155+0000","last_clean":"2023-06-23T18:03:42.924155+0000","last_became_active":"2023-06-23T18:03:37.376829+0000","last_became_peered":"2023-06-23T18:03:37.376829+0000","last_unstale":"2023-06-23T18:03:42.924155+0000","last_undegraded":"2023-06-23T18:03:42.924155+0000","last_fullsized":"2023-06-23T18:03:42.924155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157618+0000","last_change":"2023-06-23T18:03:37.754592+0000","last_active":"2023-06-23T18:03:42.157618+0000","last_peered":"2023-06-23T18:03:42.157618+0000","last_clean":"2023-06-23T18:03:42.157618+0000","last_became_active":"2023-06-23T18:03:37.754279+0000","last_became_peered":"2023-06-23T18:03:37.754279+0000","last_unstale":"2023-06-23T18:03:42.157618+0000","last_undegraded":"2023-06-23T18:03:42.157618+0000","last_fullsized":"2023-06-23T18:03:42.157618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:05:29.058510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920916+0000","last_change":"2023-06-23T18:03:41.787327+0000","last_active":"2023-06-23T18:03:42.920916+0000","last_peered":"2023-06-23T18:03:42.920916+0000","last_clean":"2023-06-23T18:03:42.920916+0000","last_became_active":"2023-06-23T18:03:41.786664+0000","last_became_peered":"2023-06-23T18:03:41.786664+0000","last_unstale":"2023-06-23T18:03:42.920916+0000","last_undegraded":"2023-06-23T18:03:42.920916+0000","last_fullsized":"2023-06-23T18:03:42.920916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922619+0000","last_change":"2023-06-23T18:03:37.377468+0000","last_active":"2023-06-23T18:03:42.922619+0000","last_peered":"2023-06-23T18:03:42.922619+0000","last_clean":"2023-06-23T18:03:42.922619+0000","last_became_active":"2023-06-23T18:03:37.376653+0000","last_became_peered":"2023-06-23T18:03:37.376653+0000","last_unstale":"2023-06-23T18:03:42.922619+0000","last_undegraded":"2023-06-23T18:03:42.922619+0000","last_fullsized":"2023-06-23T18:03:42.922619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157926+0000","last_change":"2023-06-23T18:03:37.761926+0000","last_active":"2023-06-23T18:03:42.157926+0000","last_peered":"2023-06-23T18:03:42.157926+0000","last_clean":"2023-06-23T18:03:42.157926+0000","last_became_active":"2023-06-23T18:03:37.761621+0000","last_became_peered":"2023-06-23T18:03:37.761621+0000","last_unstale":"2023-06-23T18:03:42.157926+0000","last_undegraded":"2023-06-23T18:03:42.157926+0000","last_fullsized":"2023-06-23T18:03:42.157926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:20.322939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918838+0000","last_change":"2023-06-23T18:03:41.782917+0000","last_active":"2023-06-23T18:03:42.918838+0000","last_peered":"2023-06-23T18:03:42.918838+0000","last_clean":"2023-06-23T18:03:42.918838+0000","last_became_active":"2023-06-23T18:03:41.782516+0000","last_became_peered":"2023-06-23T18:03:41.782516+0000","last_unstale":"2023-06-23T18:03:42.918838+0000","last_undegraded":"2023-06-23T18:03:42.918838+0000","last_fullsized":"2023-06-23T18:03:42.918838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920996+0000","last_change":"2023-06-23T18:03:37.379511+0000","last_active":"2023-06-23T18:03:42.920996+0000","last_peered":"2023-06-23T18:03:42.920996+0000","last_clean":"2023-06-23T18:03:42.920996+0000","last_became_active":"2023-06-23T18:03:37.379252+0000","last_became_peered":"2023-06-23T18:03:37.379252+0000","last_unstale":"2023-06-23T18:03:42.920996+0000","last_undegraded":"2023-06-23T18:03:42.920996+0000","last_fullsized":"2023-06-23T18:03:42.920996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:58.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153021+0000","last_change":"2023-06-23T18:03:37.768862+0000","last_active":"2023-06-23T18:03:42.153021+0000","last_peered":"2023-06-23T18:03:42.153021+0000","last_clean":"2023-06-23T18:03:42.153021+0000","last_became_active":"2023-06-23T18:03:37.768610+0000","last_became_peered":"2023-06-23T18:03:37.768610+0000","last_unstale":"2023-06-23T18:03:42.153021+0000","last_undegraded":"2023-06-23T18:03:42.153021+0000","last_fullsized":"2023-06-23T18:03:42.153021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:10:03.453386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158181+0000","last_change":"2023-06-23T18:03:41.781852+0000","last_active":"2023-06-23T18:03:42.158181+0000","last_peered":"2023-06-23T18:03:42.158181+0000","last_clean":"2023-06-23T18:03:42.158181+0000","last_became_active":"2023-06-23T18:03:41.781321+0000","last_became_peered":"2023-06-23T18:03:41.781321+0000","last_unstale":"2023-06-23T18:03:42.158181+0000","last_undegraded":"2023-06-23T18:03:42.158181+0000","last_fullsized":"2023-06-23T18:03:42.158181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:21:44.946317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157942+0000","last_change":"2023-06-23T18:03:37.372568+0000","last_active":"2023-06-23T18:03:42.157942+0000","last_peered":"2023-06-23T18:03:42.157942+0000","last_clean":"2023-06-23T18:03:42.157942+0000","last_became_active":"2023-06-23T18:03:37.372256+0000","last_became_peered":"2023-06-23T18:03:37.372256+0000","last_unstale":"2023-06-23T18:03:42.157942+0000","last_undegraded":"2023-06-23T18:03:42.157942+0000","last_fullsized":"2023-06-23T18:03:42.157942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:01:27.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919391+0000","last_change":"2023-06-23T18:03:37.761387+0000","last_active":"2023-06-23T18:03:42.919391+0000","last_peered":"2023-06-23T18:03:42.919391+0000","last_clean":"2023-06-23T18:03:42.919391+0000","last_became_active":"2023-06-23T18:03:37.758859+0000","last_became_peered":"2023-06-23T18:03:37.758859+0000","last_unstale":"2023-06-23T18:03:42.919391+0000","last_undegraded":"2023-06-23T18:03:42.919391+0000","last_fullsized":"2023-06-23T18:03:42.919391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919777+0000","last_change":"2023-06-23T18:03:41.782177+0000","last_active":"2023-06-23T18:03:42.919777+0000","last_peered":"2023-06-23T18:03:42.919777+0000","last_clean":"2023-06-23T18:03:42.919777+0000","last_became_active":"2023-06-23T18:03:41.781457+0000","last_became_peered":"2023-06-23T18:03:41.781457+0000","last_unstale":"2023-06-23T18:03:42.919777+0000","last_undegraded":"2023-06-23T18:03:42.919777+0000","last_fullsized":"2023-06-23T18:03:42.919777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:38:17.449084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916697+0000","last_change":"2023-06-23T18:03:37.367776+0000","last_active":"2023-06-23T18:03:42.916697+0000","last_peered":"2023-06-23T18:03:42.916697+0000","last_clean":"2023-06-23T18:03:42.916697+0000","last_became_active":"2023-06-23T18:03:37.367288+0000","last_became_peered":"2023-06-23T18:03:37.367288+0000","last_unstale":"2023-06-23T18:03:42.916697+0000","last_undegraded":"2023-06-23T18:03:42.916697+0000","last_fullsized":"2023-06-23T18:03:42.916697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:46.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922187+0000","last_change":"2023-06-23T18:03:37.769820+0000","last_active":"2023-06-23T18:03:42.922187+0000","last_peered":"2023-06-23T18:03:42.922187+0000","last_clean":"2023-06-23T18:03:42.922187+0000","last_became_active":"2023-06-23T18:03:37.769571+0000","last_became_peered":"2023-06-23T18:03:37.769571+0000","last_unstale":"2023-06-23T18:03:42.922187+0000","last_undegraded":"2023-06-23T18:03:42.922187+0000","last_fullsized":"2023-06-23T18:03:42.922187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921931+0000","last_change":"2023-06-23T18:03:41.783182+0000","last_active":"2023-06-23T18:03:42.921931+0000","last_peered":"2023-06-23T18:03:42.921931+0000","last_clean":"2023-06-23T18:03:42.921931+0000","last_became_active":"2023-06-23T18:03:41.782753+0000","last_became_peered":"2023-06-23T18:03:41.782753+0000","last_unstale":"2023-06-23T18:03:42.921931+0000","last_undegraded":"2023-06-23T18:03:42.921931+0000","last_fullsized":"2023-06-23T18:03:42.921931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916193+0000","last_change":"2023-06-23T18:03:37.378552+0000","last_active":"2023-06-23T18:03:42.916193+0000","last_peered":"2023-06-23T18:03:42.916193+0000","last_clean":"2023-06-23T18:03:42.916193+0000","last_became_active":"2023-06-23T18:03:37.378264+0000","last_became_peered":"2023-06-23T18:03:37.378264+0000","last_unstale":"2023-06-23T18:03:42.916193+0000","last_undegraded":"2023-06-23T18:03:42.916193+0000","last_fullsized":"2023-06-23T18:03:42.916193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:04:27.977965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920059+0000","last_change":"2023-06-23T18:03:37.755621+0000","last_active":"2023-06-23T18:03:42.920059+0000","last_peered":"2023-06-23T18:03:42.920059+0000","last_clean":"2023-06-23T18:03:42.920059+0000","last_became_active":"2023-06-23T18:03:37.755341+0000","last_became_peered":"2023-06-23T18:03:37.755341+0000","last_unstale":"2023-06-23T18:03:42.920059+0000","last_undegraded":"2023-06-23T18:03:42.920059+0000","last_fullsized":"2023-06-23T18:03:42.920059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:11:22.805985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915863+0000","last_change":"2023-06-23T18:03:37.758569+0000","last_active":"2023-06-23T18:03:42.915863+0000","last_peered":"2023-06-23T18:03:42.915863+0000","last_clean":"2023-06-23T18:03:42.915863+0000","last_became_active":"2023-06-23T18:03:37.758229+0000","last_became_peered":"2023-06-23T18:03:37.758229+0000","last_unstale":"2023-06-23T18:03:42.915863+0000","last_undegraded":"2023-06-23T18:03:42.915863+0000","last_fullsized":"2023-06-23T18:03:42.915863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:56:28.950360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922336+0000","last_change":"2023-06-23T18:03:37.377384+0000","last_active":"2023-06-23T18:03:42.922336+0000","last_peered":"2023-06-23T18:03:42.922336+0000","last_clean":"2023-06-23T18:03:42.922336+0000","last_became_active":"2023-06-23T18:03:37.376940+0000","last_became_peered":"2023-06-23T18:03:37.376940+0000","last_unstale":"2023-06-23T18:03:42.922336+0000","last_undegraded":"2023-06-23T18:03:42.922336+0000","last_fullsized":"2023-06-23T18:03:42.922336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919105+0000","last_change":"2023-06-23T18:03:41.787485+0000","last_active":"2023-06-23T18:03:42.919105+0000","last_peered":"2023-06-23T18:03:42.919105+0000","last_clean":"2023-06-23T18:03:42.919105+0000","last_became_active":"2023-06-23T18:03:41.787017+0000","last_became_peered":"2023-06-23T18:03:41.787017+0000","last_unstale":"2023-06-23T18:03:42.919105+0000","last_undegraded":"2023-06-23T18:03:42.919105+0000","last_fullsized":"2023-06-23T18:03:42.919105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:48:29.879084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916767+0000","last_change":"2023-06-23T18:03:37.768665+0000","last_active":"2023-06-23T18:03:42.916767+0000","last_peered":"2023-06-23T18:03:42.916767+0000","last_clean":"2023-06-23T18:03:42.916767+0000","last_became_active":"2023-06-23T18:03:37.768373+0000","last_became_peered":"2023-06-23T18:03:37.768373+0000","last_unstale":"2023-06-23T18:03:42.916767+0000","last_undegraded":"2023-06-23T18:03:42.916767+0000","last_fullsized":"2023-06-23T18:03:42.916767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:03:33.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923756+0000","last_change":"2023-06-23T18:03:37.376353+0000","last_active":"2023-06-23T18:03:42.923756+0000","last_peered":"2023-06-23T18:03:42.923756+0000","last_clean":"2023-06-23T18:03:42.923756+0000","last_became_active":"2023-06-23T18:03:37.375931+0000","last_became_peered":"2023-06-23T18:03:37.375931+0000","last_unstale":"2023-06-23T18:03:42.923756+0000","last_undegraded":"2023-06-23T18:03:42.923756+0000","last_fullsized":"2023-06-23T18:03:42.923756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:22.517180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920338+0000","last_change":"2023-06-23T18:03:41.787494+0000","last_active":"2023-06-23T18:03:42.920338+0000","last_peered":"2023-06-23T18:03:42.920338+0000","last_clean":"2023-06-23T18:03:42.920338+0000","last_became_active":"2023-06-23T18:03:41.787099+0000","last_became_peered":"2023-06-23T18:03:41.787099+0000","last_unstale":"2023-06-23T18:03:42.920338+0000","last_undegraded":"2023-06-23T18:03:42.920338+0000","last_fullsized":"2023-06-23T18:03:42.920338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922054+0000","last_change":"2023-06-23T18:03:37.377640+0000","last_active":"2023-06-23T18:03:42.922054+0000","last_peered":"2023-06-23T18:03:42.922054+0000","last_clean":"2023-06-23T18:03:42.922054+0000","last_became_active":"2023-06-23T18:03:37.377010+0000","last_became_peered":"2023-06-23T18:03:37.377010+0000","last_unstale":"2023-06-23T18:03:42.922054+0000","last_undegraded":"2023-06-23T18:03:42.922054+0000","last_fullsized":"2023-06-23T18:03:42.922054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917168+0000","last_change":"2023-06-23T18:03:37.756104+0000","last_active":"2023-06-23T18:03:42.917168+0000","last_peered":"2023-06-23T18:03:42.917168+0000","last_clean":"2023-06-23T18:03:42.917168+0000","last_became_active":"2023-06-23T18:03:37.755767+0000","last_became_peered":"2023-06-23T18:03:37.755767+0000","last_unstale":"2023-06-23T18:03:42.917168+0000","last_undegraded":"2023-06-23T18:03:42.917168+0000","last_fullsized":"2023-06-23T18:03:42.917168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:22:59.783746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917308+0000","last_change":"2023-06-23T18:03:41.785136+0000","last_active":"2023-06-23T18:03:42.917308+0000","last_peered":"2023-06-23T18:03:42.917308+0000","last_clean":"2023-06-23T18:03:42.917308+0000","last_became_active":"2023-06-23T18:03:41.784711+0000","last_became_peered":"2023-06-23T18:03:41.784711+0000","last_unstale":"2023-06-23T18:03:42.917308+0000","last_undegraded":"2023-06-23T18:03:42.917308+0000","last_fullsized":"2023-06-23T18:03:42.917308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:47.124821+0000","last_change":"2023-06-23T18:03:37.376017+0000","last_active":"2023-06-23T18:03:47.124821+0000","last_peered":"2023-06-23T18:03:47.124821+0000","last_clean":"2023-06-23T18:03:47.124821+0000","last_became_active":"2023-06-23T18:03:37.375728+0000","last_became_peered":"2023-06-23T18:03:37.375728+0000","last_unstale":"2023-06-23T18:03:47.124821+0000","last_undegraded":"2023-06-23T18:03:47.124821+0000","last_fullsized":"2023-06-23T18:03:47.124821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:57.309841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158169+0000","last_change":"2023-06-23T18:03:37.752163+0000","last_active":"2023-06-23T18:03:42.158169+0000","last_peered":"2023-06-23T18:03:42.158169+0000","last_clean":"2023-06-23T18:03:42.158169+0000","last_became_active":"2023-06-23T18:03:37.751393+0000","last_became_peered":"2023-06-23T18:03:37.751393+0000","last_unstale":"2023-06-23T18:03:42.158169+0000","last_undegraded":"2023-06-23T18:03:42.158169+0000","last_fullsized":"2023-06-23T18:03:42.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:54:13.905688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916838+0000","last_change":"2023-06-23T18:03:41.790305+0000","last_active":"2023-06-23T18:03:42.916838+0000","last_peered":"2023-06-23T18:03:42.916838+0000","last_clean":"2023-06-23T18:03:42.916838+0000","last_became_active":"2023-06-23T18:03:41.787857+0000","last_became_peered":"2023-06-23T18:03:41.787857+0000","last_unstale":"2023-06-23T18:03:42.916838+0000","last_undegraded":"2023-06-23T18:03:42.916838+0000","last_fullsized":"2023-06-23T18:03:42.916838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:54.379340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157126+0000","last_change":"2023-06-23T18:03:37.751336+0000","last_active":"2023-06-23T18:03:42.157126+0000","last_peered":"2023-06-23T18:03:42.157126+0000","last_clean":"2023-06-23T18:03:42.157126+0000","last_became_active":"2023-06-23T18:03:37.750981+0000","last_became_peered":"2023-06-23T18:03:37.750981+0000","last_unstale":"2023-06-23T18:03:42.157126+0000","last_undegraded":"2023-06-23T18:03:42.157126+0000","last_fullsized":"2023-06-23T18:03:42.157126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:17:15.424492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920331+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:03:42.920331+0000","last_peered":"2023-06-23T18:03:42.920331+0000","last_clean":"2023-06-23T18:03:42.920331+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:03:42.920331+0000","last_undegraded":"2023-06-23T18:03:42.920331+0000","last_fullsized":"2023-06-23T18:03:42.920331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:49:09.922836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922412+0000","last_change":"2023-06-23T18:03:37.375755+0000","last_active":"2023-06-23T18:03:42.922412+0000","last_peered":"2023-06-23T18:03:42.922412+0000","last_clean":"2023-06-23T18:03:42.922412+0000","last_became_active":"2023-06-23T18:03:37.375421+0000","last_became_peered":"2023-06-23T18:03:37.375421+0000","last_unstale":"2023-06-23T18:03:42.922412+0000","last_undegraded":"2023-06-23T18:03:42.922412+0000","last_fullsized":"2023-06-23T18:03:42.922412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915864+0000","last_change":"2023-06-23T18:03:41.785345+0000","last_active":"2023-06-23T18:03:42.915864+0000","last_peered":"2023-06-23T18:03:42.915864+0000","last_clean":"2023-06-23T18:03:42.915864+0000","last_became_active":"2023-06-23T18:03:41.784105+0000","last_became_peered":"2023-06-23T18:03:41.784105+0000","last_unstale":"2023-06-23T18:03:42.915864+0000","last_undegraded":"2023-06-23T18:03:42.915864+0000","last_fullsized":"2023-06-23T18:03:42.915864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:07:51.445920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915986+0000","last_change":"2023-06-23T18:03:37.768794+0000","last_active":"2023-06-23T18:03:42.915986+0000","last_peered":"2023-06-23T18:03:42.915986+0000","last_clean":"2023-06-23T18:03:42.915986+0000","last_became_active":"2023-06-23T18:03:37.768517+0000","last_became_peered":"2023-06-23T18:03:37.768517+0000","last_unstale":"2023-06-23T18:03:42.915986+0000","last_undegraded":"2023-06-23T18:03:42.915986+0000","last_fullsized":"2023-06-23T18:03:42.915986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:53.245906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920435+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:03:42.920435+0000","last_peered":"2023-06-23T18:03:42.920435+0000","last_clean":"2023-06-23T18:03:42.920435+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:03:42.920435+0000","last_undegraded":"2023-06-23T18:03:42.920435+0000","last_fullsized":"2023-06-23T18:03:42.920435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921748+0000","last_change":"2023-06-23T18:03:37.376156+0000","last_active":"2023-06-23T18:03:42.921748+0000","last_peered":"2023-06-23T18:03:42.921748+0000","last_clean":"2023-06-23T18:03:42.921748+0000","last_became_active":"2023-06-23T18:03:37.375870+0000","last_became_peered":"2023-06-23T18:03:37.375870+0000","last_unstale":"2023-06-23T18:03:42.921748+0000","last_undegraded":"2023-06-23T18:03:42.921748+0000","last_fullsized":"2023-06-23T18:03:42.921748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156557+0000","last_change":"2023-06-23T18:03:41.783782+0000","last_active":"2023-06-23T18:03:42.156557+0000","last_peered":"2023-06-23T18:03:42.156557+0000","last_clean":"2023-06-23T18:03:42.156557+0000","last_became_active":"2023-06-23T18:03:41.783336+0000","last_became_peered":"2023-06-23T18:03:41.783336+0000","last_unstale":"2023-06-23T18:03:42.156557+0000","last_undegraded":"2023-06-23T18:03:42.156557+0000","last_fullsized":"2023-06-23T18:03:42.156557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:55.511609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920320+0000","last_change":"2023-06-23T18:03:37.755707+0000","last_active":"2023-06-23T18:03:42.920320+0000","last_peered":"2023-06-23T18:03:42.920320+0000","last_clean":"2023-06-23T18:03:42.920320+0000","last_became_active":"2023-06-23T18:03:37.755413+0000","last_became_peered":"2023-06-23T18:03:37.755413+0000","last_unstale":"2023-06-23T18:03:42.920320+0000","last_undegraded":"2023-06-23T18:03:42.920320+0000","last_fullsized":"2023-06-23T18:03:42.920320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922407+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:03:42.922407+0000","last_peered":"2023-06-23T18:03:42.922407+0000","last_clean":"2023-06-23T18:03:42.922407+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:03:42.922407+0000","last_undegraded":"2023-06-23T18:03:42.922407+0000","last_fullsized":"2023-06-23T18:03:42.922407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:55.340944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918354+0000","last_change":"2023-06-23T18:03:37.370271+0000","last_active":"2023-06-23T18:03:42.918354+0000","last_peered":"2023-06-23T18:03:42.918354+0000","last_clean":"2023-06-23T18:03:42.918354+0000","last_became_active":"2023-06-23T18:03:37.369994+0000","last_became_peered":"2023-06-23T18:03:37.369994+0000","last_unstale":"2023-06-23T18:03:42.918354+0000","last_undegraded":"2023-06-23T18:03:42.918354+0000","last_fullsized":"2023-06-23T18:03:42.918354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:36.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920193+0000","last_change":"2023-06-23T18:03:41.793568+0000","last_active":"2023-06-23T18:03:42.920193+0000","last_peered":"2023-06-23T18:03:42.920193+0000","last_clean":"2023-06-23T18:03:42.920193+0000","last_became_active":"2023-06-23T18:03:41.793190+0000","last_became_peered":"2023-06-23T18:03:41.793190+0000","last_unstale":"2023-06-23T18:03:42.920193+0000","last_undegraded":"2023-06-23T18:03:42.920193+0000","last_fullsized":"2023-06-23T18:03:42.920193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:35.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917232+0000","last_change":"2023-06-23T18:03:37.764627+0000","last_active":"2023-06-23T18:03:42.917232+0000","last_peered":"2023-06-23T18:03:42.917232+0000","last_clean":"2023-06-23T18:03:42.917232+0000","last_became_active":"2023-06-23T18:03:37.764255+0000","last_became_peered":"2023-06-23T18:03:37.764255+0000","last_unstale":"2023-06-23T18:03:42.917232+0000","last_undegraded":"2023-06-23T18:03:42.917232+0000","last_fullsized":"2023-06-23T18:03:42.917232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918569+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:03:42.918569+0000","last_peered":"2023-06-23T18:03:42.918569+0000","last_clean":"2023-06-23T18:03:42.918569+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:03:42.918569+0000","last_undegraded":"2023-06-23T18:03:42.918569+0000","last_fullsized":"2023-06-23T18:03:42.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:25.233307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924293+0000","last_change":"2023-06-23T18:03:37.371127+0000","last_active":"2023-06-23T18:03:42.924293+0000","last_peered":"2023-06-23T18:03:42.924293+0000","last_clean":"2023-06-23T18:03:42.924293+0000","last_became_active":"2023-06-23T18:03:37.370810+0000","last_became_peered":"2023-06-23T18:03:37.370810+0000","last_unstale":"2023-06-23T18:03:42.924293+0000","last_undegraded":"2023-06-23T18:03:42.924293+0000","last_fullsized":"2023-06-23T18:03:42.924293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:42.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919827+0000","last_change":"2023-06-23T18:03:41.791645+0000","last_active":"2023-06-23T18:03:42.919827+0000","last_peered":"2023-06-23T18:03:42.919827+0000","last_clean":"2023-06-23T18:03:42.919827+0000","last_became_active":"2023-06-23T18:03:41.791276+0000","last_became_peered":"2023-06-23T18:03:41.791276+0000","last_unstale":"2023-06-23T18:03:42.919827+0000","last_undegraded":"2023-06-23T18:03:42.919827+0000","last_fullsized":"2023-06-23T18:03:42.919827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:24.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918088+0000","last_change":"2023-06-23T18:03:37.754565+0000","last_active":"2023-06-23T18:03:42.918088+0000","last_peered":"2023-06-23T18:03:42.918088+0000","last_clean":"2023-06-23T18:03:42.918088+0000","last_became_active":"2023-06-23T18:03:37.754281+0000","last_became_peered":"2023-06-23T18:03:37.754281+0000","last_unstale":"2023-06-23T18:03:42.918088+0000","last_undegraded":"2023-06-23T18:03:42.918088+0000","last_fullsized":"2023-06-23T18:03:42.918088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158267+0000","last_change":"2023-06-23T18:03:37.381149+0000","last_active":"2023-06-23T18:03:42.158267+0000","last_peered":"2023-06-23T18:03:42.158267+0000","last_clean":"2023-06-23T18:03:42.158267+0000","last_became_active":"2023-06-23T18:03:37.380831+0000","last_became_peered":"2023-06-23T18:03:37.380831+0000","last_unstale":"2023-06-23T18:03:42.158267+0000","last_undegraded":"2023-06-23T18:03:42.158267+0000","last_fullsized":"2023-06-23T18:03:42.158267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:40:27.306991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923558+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:03:42.923558+0000","last_peered":"2023-06-23T18:03:42.923558+0000","last_clean":"2023-06-23T18:03:42.923558+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:03:42.923558+0000","last_undegraded":"2023-06-23T18:03:42.923558+0000","last_fullsized":"2023-06-23T18:03:42.923558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921567+0000","last_change":"2023-06-23T18:03:41.787633+0000","last_active":"2023-06-23T18:03:42.921567+0000","last_peered":"2023-06-23T18:03:42.921567+0000","last_clean":"2023-06-23T18:03:42.921567+0000","last_became_active":"2023-06-23T18:03:41.786906+0000","last_became_peered":"2023-06-23T18:03:41.786906+0000","last_unstale":"2023-06-23T18:03:42.921567+0000","last_undegraded":"2023-06-23T18:03:42.921567+0000","last_fullsized":"2023-06-23T18:03:42.921567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919645+0000","last_change":"2023-06-23T18:03:37.764949+0000","last_active":"2023-06-23T18:03:42.919645+0000","last_peered":"2023-06-23T18:03:42.919645+0000","last_clean":"2023-06-23T18:03:42.919645+0000","last_became_active":"2023-06-23T18:03:37.764358+0000","last_became_peered":"2023-06-23T18:03:37.764358+0000","last_unstale":"2023-06-23T18:03:42.919645+0000","last_undegraded":"2023-06-23T18:03:42.919645+0000","last_fullsized":"2023-06-23T18:03:42.919645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916086+0000","last_change":"2023-06-23T18:03:37.371283+0000","last_active":"2023-06-23T18:03:42.916086+0000","last_peered":"2023-06-23T18:03:42.916086+0000","last_clean":"2023-06-23T18:03:42.916086+0000","last_became_active":"2023-06-23T18:03:37.370358+0000","last_became_peered":"2023-06-23T18:03:37.370358+0000","last_unstale":"2023-06-23T18:03:42.916086+0000","last_undegraded":"2023-06-23T18:03:42.916086+0000","last_fullsized":"2023-06-23T18:03:42.916086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152204+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:03:42.152204+0000","last_peered":"2023-06-23T18:03:42.152204+0000","last_clean":"2023-06-23T18:03:42.152204+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:03:42.152204+0000","last_undegraded":"2023-06-23T18:03:42.152204+0000","last_fullsized":"2023-06-23T18:03:42.152204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:44.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152495+0000","last_change":"2023-06-23T18:03:41.787323+0000","last_active":"2023-06-23T18:03:42.152495+0000","last_peered":"2023-06-23T18:03:42.152495+0000","last_clean":"2023-06-23T18:03:42.152495+0000","last_became_active":"2023-06-23T18:03:41.786742+0000","last_became_peered":"2023-06-23T18:03:41.786742+0000","last_unstale":"2023-06-23T18:03:42.152495+0000","last_undegraded":"2023-06-23T18:03:42.152495+0000","last_fullsized":"2023-06-23T18:03:42.152495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:39:47.931343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922227+0000","last_change":"2023-06-23T18:03:37.743506+0000","last_active":"2023-06-23T18:03:42.922227+0000","last_peered":"2023-06-23T18:03:42.922227+0000","last_clean":"2023-06-23T18:03:42.922227+0000","last_became_active":"2023-06-23T18:03:37.743222+0000","last_became_peered":"2023-06-23T18:03:37.743222+0000","last_unstale":"2023-06-23T18:03:42.922227+0000","last_undegraded":"2023-06-23T18:03:42.922227+0000","last_fullsized":"2023-06-23T18:03:42.922227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:43.133522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920271+0000","last_change":"2023-06-23T18:03:37.375782+0000","last_active":"2023-06-23T18:03:42.920271+0000","last_peered":"2023-06-23T18:03:42.920271+0000","last_clean":"2023-06-23T18:03:42.920271+0000","last_became_active":"2023-06-23T18:03:37.375451+0000","last_became_peered":"2023-06-23T18:03:37.375451+0000","last_unstale":"2023-06-23T18:03:42.920271+0000","last_undegraded":"2023-06-23T18:03:42.920271+0000","last_fullsized":"2023-06-23T18:03:42.920271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:35.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921282+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:03:42.921282+0000","last_peered":"2023-06-23T18:03:42.921282+0000","last_clean":"2023-06-23T18:03:42.921282+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:03:42.921282+0000","last_undegraded":"2023-06-23T18:03:42.921282+0000","last_fullsized":"2023-06-23T18:03:42.921282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922998+0000","last_change":"2023-06-23T18:03:41.788944+0000","last_active":"2023-06-23T18:03:42.922998+0000","last_peered":"2023-06-23T18:03:42.922998+0000","last_clean":"2023-06-23T18:03:42.922998+0000","last_became_active":"2023-06-23T18:03:41.785916+0000","last_became_peered":"2023-06-23T18:03:41.785916+0000","last_unstale":"2023-06-23T18:03:42.922998+0000","last_undegraded":"2023-06-23T18:03:42.922998+0000","last_fullsized":"2023-06-23T18:03:42.922998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919919+0000","last_change":"2023-06-23T18:03:37.750749+0000","last_active":"2023-06-23T18:03:42.919919+0000","last_peered":"2023-06-23T18:03:42.919919+0000","last_clean":"2023-06-23T18:03:42.919919+0000","last_became_active":"2023-06-23T18:03:37.750424+0000","last_became_peered":"2023-06-23T18:03:37.750424+0000","last_unstale":"2023-06-23T18:03:42.919919+0000","last_undegraded":"2023-06-23T18:03:42.919919+0000","last_fullsized":"2023-06-23T18:03:42.919919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:10.920865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917678+0000","last_change":"2023-06-23T18:03:37.377053+0000","last_active":"2023-06-23T18:03:42.917678+0000","last_peered":"2023-06-23T18:03:42.917678+0000","last_clean":"2023-06-23T18:03:42.917678+0000","last_became_active":"2023-06-23T18:03:37.376770+0000","last_became_peered":"2023-06-23T18:03:37.376770+0000","last_unstale":"2023-06-23T18:03:42.917678+0000","last_undegraded":"2023-06-23T18:03:42.917678+0000","last_fullsized":"2023-06-23T18:03:42.917678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923092+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:03:42.923092+0000","last_peered":"2023-06-23T18:03:42.923092+0000","last_clean":"2023-06-23T18:03:42.923092+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:03:42.923092+0000","last_undegraded":"2023-06-23T18:03:42.923092+0000","last_fullsized":"2023-06-23T18:03:42.923092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:00.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921588+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:03:42.921588+0000","last_peered":"2023-06-23T18:03:42.921588+0000","last_clean":"2023-06-23T18:03:42.921588+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:03:42.921588+0000","last_undegraded":"2023-06-23T18:03:42.921588+0000","last_fullsized":"2023-06-23T18:03:42.921588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921628+0000","last_change":"2023-06-23T18:03:41.792372+0000","last_active":"2023-06-23T18:03:42.921628+0000","last_peered":"2023-06-23T18:03:42.921628+0000","last_clean":"2023-06-23T18:03:42.921628+0000","last_became_active":"2023-06-23T18:03:41.791937+0000","last_became_peered":"2023-06-23T18:03:41.791937+0000","last_unstale":"2023-06-23T18:03:42.921628+0000","last_undegraded":"2023-06-23T18:03:42.921628+0000","last_fullsized":"2023-06-23T18:03:42.921628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:26.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919683+0000","last_change":"2023-06-23T18:03:37.764721+0000","last_active":"2023-06-23T18:03:42.919683+0000","last_peered":"2023-06-23T18:03:42.919683+0000","last_clean":"2023-06-23T18:03:42.919683+0000","last_became_active":"2023-06-23T18:03:37.755578+0000","last_became_peered":"2023-06-23T18:03:37.755578+0000","last_unstale":"2023-06-23T18:03:42.919683+0000","last_undegraded":"2023-06-23T18:03:42.919683+0000","last_fullsized":"2023-06-23T18:03:42.919683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916131+0000","last_change":"2023-06-23T18:03:37.369824+0000","last_active":"2023-06-23T18:03:42.916131+0000","last_peered":"2023-06-23T18:03:42.916131+0000","last_clean":"2023-06-23T18:03:42.916131+0000","last_became_active":"2023-06-23T18:03:37.369094+0000","last_became_peered":"2023-06-23T18:03:37.369094+0000","last_unstale":"2023-06-23T18:03:42.916131+0000","last_undegraded":"2023-06-23T18:03:42.916131+0000","last_fullsized":"2023-06-23T18:03:42.916131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:53.895398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159093+0000","last_change":"2023-06-23T18:03:41.789799+0000","last_active":"2023-06-23T18:03:42.159093+0000","last_peered":"2023-06-23T18:03:42.159093+0000","last_clean":"2023-06-23T18:03:42.159093+0000","last_became_active":"2023-06-23T18:03:41.789426+0000","last_became_peered":"2023-06-23T18:03:41.789426+0000","last_unstale":"2023-06-23T18:03:42.159093+0000","last_undegraded":"2023-06-23T18:03:42.159093+0000","last_fullsized":"2023-06-23T18:03:42.159093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:36:43.668342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159103+0000","last_change":"2023-06-23T18:03:37.751763+0000","last_active":"2023-06-23T18:03:42.159103+0000","last_peered":"2023-06-23T18:03:42.159103+0000","last_clean":"2023-06-23T18:03:42.159103+0000","last_became_active":"2023-06-23T18:03:37.751310+0000","last_became_peered":"2023-06-23T18:03:37.751310+0000","last_unstale":"2023-06-23T18:03:42.159103+0000","last_undegraded":"2023-06-23T18:03:42.159103+0000","last_fullsized":"2023-06-23T18:03:42.159103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:00.581207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157189+0000","last_change":"2023-06-23T18:03:37.373425+0000","last_active":"2023-06-23T18:03:42.157189+0000","last_peered":"2023-06-23T18:03:42.157189+0000","last_clean":"2023-06-23T18:03:42.157189+0000","last_became_active":"2023-06-23T18:03:37.373154+0000","last_became_peered":"2023-06-23T18:03:37.373154+0000","last_unstale":"2023-06-23T18:03:42.157189+0000","last_undegraded":"2023-06-23T18:03:42.157189+0000","last_fullsized":"2023-06-23T18:03:42.157189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:33.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922639+0000","last_change":"2023-06-23T18:03:41.792127+0000","last_active":"2023-06-23T18:03:42.922639+0000","last_peered":"2023-06-23T18:03:42.922639+0000","last_clean":"2023-06-23T18:03:42.922639+0000","last_became_active":"2023-06-23T18:03:41.789481+0000","last_became_peered":"2023-06-23T18:03:41.789481+0000","last_unstale":"2023-06-23T18:03:42.922639+0000","last_undegraded":"2023-06-23T18:03:42.922639+0000","last_fullsized":"2023-06-23T18:03:42.922639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158754+0000","last_change":"2023-06-23T18:03:37.754662+0000","last_active":"2023-06-23T18:03:42.158754+0000","last_peered":"2023-06-23T18:03:42.158754+0000","last_clean":"2023-06-23T18:03:42.158754+0000","last_became_active":"2023-06-23T18:03:37.753982+0000","last_became_peered":"2023-06-23T18:03:37.753982+0000","last_unstale":"2023-06-23T18:03:42.158754+0000","last_undegraded":"2023-06-23T18:03:42.158754+0000","last_fullsized":"2023-06-23T18:03:42.158754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:40.534179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921128+0000","last_change":"2023-06-23T18:03:37.380222+0000","last_active":"2023-06-23T18:03:42.921128+0000","last_peered":"2023-06-23T18:03:42.921128+0000","last_clean":"2023-06-23T18:03:42.921128+0000","last_became_active":"2023-06-23T18:03:37.379866+0000","last_became_peered":"2023-06-23T18:03:37.379866+0000","last_unstale":"2023-06-23T18:03:42.921128+0000","last_undegraded":"2023-06-23T18:03:42.921128+0000","last_fullsized":"2023-06-23T18:03:42.921128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:12.333579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917855+0000","last_change":"2023-06-23T18:03:41.782310+0000","last_active":"2023-06-23T18:03:42.917855+0000","last_peered":"2023-06-23T18:03:42.917855+0000","last_clean":"2023-06-23T18:03:42.917855+0000","last_became_active":"2023-06-23T18:03:41.781820+0000","last_became_peered":"2023-06-23T18:03:41.781820+0000","last_unstale":"2023-06-23T18:03:42.917855+0000","last_undegraded":"2023-06-23T18:03:42.917855+0000","last_fullsized":"2023-06-23T18:03:42.917855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:39.845448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158787+0000","last_change":"2023-06-23T18:03:37.753742+0000","last_active":"2023-06-23T18:03:42.158787+0000","last_peered":"2023-06-23T18:03:42.158787+0000","last_clean":"2023-06-23T18:03:42.158787+0000","last_became_active":"2023-06-23T18:03:37.753391+0000","last_became_peered":"2023-06-23T18:03:37.753391+0000","last_unstale":"2023-06-23T18:03:42.158787+0000","last_undegraded":"2023-06-23T18:03:42.158787+0000","last_fullsized":"2023-06-23T18:03:42.158787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:54.787520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918436+0000","last_change":"2023-06-23T18:03:37.378501+0000","last_active":"2023-06-23T18:03:42.918436+0000","last_peered":"2023-06-23T18:03:42.918436+0000","last_clean":"2023-06-23T18:03:42.918436+0000","last_became_active":"2023-06-23T18:03:37.378222+0000","last_became_peered":"2023-06-23T18:03:37.378222+0000","last_unstale":"2023-06-23T18:03:42.918436+0000","last_undegraded":"2023-06-23T18:03:42.918436+0000","last_fullsized":"2023-06-23T18:03:42.918436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:49.965045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152626+0000","last_change":"2023-06-23T18:03:41.788999+0000","last_active":"2023-06-23T18:03:42.152626+0000","last_peered":"2023-06-23T18:03:42.152626+0000","last_clean":"2023-06-23T18:03:42.152626+0000","last_became_active":"2023-06-23T18:03:41.788606+0000","last_became_peered":"2023-06-23T18:03:41.788606+0000","last_unstale":"2023-06-23T18:03:42.152626+0000","last_undegraded":"2023-06-23T18:03:42.152626+0000","last_fullsized":"2023-06-23T18:03:42.152626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:40:16.265229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920805+0000","last_change":"2023-06-23T18:03:37.751267+0000","last_active":"2023-06-23T18:03:42.920805+0000","last_peered":"2023-06-23T18:03:42.920805+0000","last_clean":"2023-06-23T18:03:42.920805+0000","last_became_active":"2023-06-23T18:03:37.746037+0000","last_became_peered":"2023-06-23T18:03:37.746037+0000","last_unstale":"2023-06-23T18:03:42.920805+0000","last_undegraded":"2023-06-23T18:03:42.920805+0000","last_fullsized":"2023-06-23T18:03:42.920805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922834+0000","last_change":"2023-06-23T18:03:37.369505+0000","last_active":"2023-06-23T18:03:42.922834+0000","last_peered":"2023-06-23T18:03:42.922834+0000","last_clean":"2023-06-23T18:03:42.922834+0000","last_became_active":"2023-06-23T18:03:37.369174+0000","last_became_peered":"2023-06-23T18:03:37.369174+0000","last_unstale":"2023-06-23T18:03:42.922834+0000","last_undegraded":"2023-06-23T18:03:42.922834+0000","last_fullsized":"2023-06-23T18:03:42.922834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151984+0000","last_change":"2023-06-23T18:03:41.781321+0000","last_active":"2023-06-23T18:03:42.151984+0000","last_peered":"2023-06-23T18:03:42.151984+0000","last_clean":"2023-06-23T18:03:42.151984+0000","last_became_active":"2023-06-23T18:03:41.780831+0000","last_became_peered":"2023-06-23T18:03:41.780831+0000","last_unstale":"2023-06-23T18:03:42.151984+0000","last_undegraded":"2023-06-23T18:03:42.151984+0000","last_fullsized":"2023-06-23T18:03:42.151984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:53.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920510+0000","last_change":"2023-06-23T18:03:37.761108+0000","last_active":"2023-06-23T18:03:42.920510+0000","last_peered":"2023-06-23T18:03:42.920510+0000","last_clean":"2023-06-23T18:03:42.920510+0000","last_became_active":"2023-06-23T18:03:37.760041+0000","last_became_peered":"2023-06-23T18:03:37.760041+0000","last_unstale":"2023-06-23T18:03:42.920510+0000","last_undegraded":"2023-06-23T18:03:42.920510+0000","last_fullsized":"2023-06-23T18:03:42.920510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:46:38.396933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922050+0000","last_change":"2023-06-23T18:03:37.377838+0000","last_active":"2023-06-23T18:03:42.922050+0000","last_peered":"2023-06-23T18:03:42.922050+0000","last_clean":"2023-06-23T18:03:42.922050+0000","last_became_active":"2023-06-23T18:03:37.377381+0000","last_became_peered":"2023-06-23T18:03:37.377381+0000","last_unstale":"2023-06-23T18:03:42.922050+0000","last_undegraded":"2023-06-23T18:03:42.922050+0000","last_fullsized":"2023-06-23T18:03:42.922050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152297+0000","last_change":"2023-06-23T18:03:41.787926+0000","last_active":"2023-06-23T18:03:42.152297+0000","last_peered":"2023-06-23T18:03:42.152297+0000","last_clean":"2023-06-23T18:03:42.152297+0000","last_became_active":"2023-06-23T18:03:41.787539+0000","last_became_peered":"2023-06-23T18:03:41.787539+0000","last_unstale":"2023-06-23T18:03:42.152297+0000","last_undegraded":"2023-06-23T18:03:42.152297+0000","last_fullsized":"2023-06-23T18:03:42.152297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:49.966258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156824+0000","last_change":"2023-06-23T18:03:37.745296+0000","last_active":"2023-06-23T18:03:42.156824+0000","last_peered":"2023-06-23T18:03:42.156824+0000","last_clean":"2023-06-23T18:03:42.156824+0000","last_became_active":"2023-06-23T18:03:37.744996+0000","last_became_peered":"2023-06-23T18:03:37.744996+0000","last_unstale":"2023-06-23T18:03:42.156824+0000","last_undegraded":"2023-06-23T18:03:42.156824+0000","last_fullsized":"2023-06-23T18:03:42.156824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:07:37.054289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922271+0000","last_change":"2023-06-23T18:03:37.381405+0000","last_active":"2023-06-23T18:03:42.922271+0000","last_peered":"2023-06-23T18:03:42.922271+0000","last_clean":"2023-06-23T18:03:42.922271+0000","last_became_active":"2023-06-23T18:03:37.381038+0000","last_became_peered":"2023-06-23T18:03:37.381038+0000","last_unstale":"2023-06-23T18:03:42.922271+0000","last_undegraded":"2023-06-23T18:03:42.922271+0000","last_fullsized":"2023-06-23T18:03:42.922271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151761+0000","last_change":"2023-06-23T18:03:41.787211+0000","last_active":"2023-06-23T18:03:42.151761+0000","last_peered":"2023-06-23T18:03:42.151761+0000","last_clean":"2023-06-23T18:03:42.151761+0000","last_became_active":"2023-06-23T18:03:41.786809+0000","last_became_peered":"2023-06-23T18:03:41.786809+0000","last_unstale":"2023-06-23T18:03:42.151761+0000","last_undegraded":"2023-06-23T18:03:42.151761+0000","last_fullsized":"2023-06-23T18:03:42.151761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:42.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919723+0000","last_change":"2023-06-23T18:03:37.770649+0000","last_active":"2023-06-23T18:03:42.919723+0000","last_peered":"2023-06-23T18:03:42.919723+0000","last_clean":"2023-06-23T18:03:42.919723+0000","last_became_active":"2023-06-23T18:03:37.770326+0000","last_became_peered":"2023-06-23T18:03:37.770326+0000","last_unstale":"2023-06-23T18:03:42.919723+0000","last_undegraded":"2023-06-23T18:03:42.919723+0000","last_fullsized":"2023-06-23T18:03:42.919723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923872+0000","last_change":"2023-06-23T18:03:37.381617+0000","last_active":"2023-06-23T18:03:42.923872+0000","last_peered":"2023-06-23T18:03:42.923872+0000","last_clean":"2023-06-23T18:03:42.923872+0000","last_became_active":"2023-06-23T18:03:37.381342+0000","last_became_peered":"2023-06-23T18:03:37.381342+0000","last_unstale":"2023-06-23T18:03:42.923872+0000","last_undegraded":"2023-06-23T18:03:42.923872+0000","last_fullsized":"2023-06-23T18:03:42.923872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:31.692075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920724+0000","last_change":"2023-06-23T18:03:41.791679+0000","last_active":"2023-06-23T18:03:42.920724+0000","last_peered":"2023-06-23T18:03:42.920724+0000","last_clean":"2023-06-23T18:03:42.920724+0000","last_became_active":"2023-06-23T18:03:41.788101+0000","last_became_peered":"2023-06-23T18:03:41.788101+0000","last_unstale":"2023-06-23T18:03:42.920724+0000","last_undegraded":"2023-06-23T18:03:42.920724+0000","last_fullsized":"2023-06-23T18:03:42.920724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924128+0000","last_change":"2023-06-23T18:03:41.789672+0000","last_active":"2023-06-23T18:03:42.924128+0000","last_peered":"2023-06-23T18:03:42.924128+0000","last_clean":"2023-06-23T18:03:42.924128+0000","last_became_active":"2023-06-23T18:03:41.789258+0000","last_became_peered":"2023-06-23T18:03:41.789258+0000","last_unstale":"2023-06-23T18:03:42.924128+0000","last_undegraded":"2023-06-23T18:03:42.924128+0000","last_fullsized":"2023-06-23T18:03:42.924128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:35:03.864692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919206+0000","last_change":"2023-06-23T18:03:37.380523+0000","last_active":"2023-06-23T18:03:42.919206+0000","last_peered":"2023-06-23T18:03:42.919206+0000","last_clean":"2023-06-23T18:03:42.919206+0000","last_became_active":"2023-06-23T18:03:37.380210+0000","last_became_peered":"2023-06-23T18:03:37.380210+0000","last_unstale":"2023-06-23T18:03:42.919206+0000","last_undegraded":"2023-06-23T18:03:42.919206+0000","last_fullsized":"2023-06-23T18:03:42.919206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:57.050399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921568+0000","last_change":"2023-06-23T18:03:37.769174+0000","last_active":"2023-06-23T18:03:42.921568+0000","last_peered":"2023-06-23T18:03:42.921568+0000","last_clean":"2023-06-23T18:03:42.921568+0000","last_became_active":"2023-06-23T18:03:37.768922+0000","last_became_peered":"2023-06-23T18:03:37.768922+0000","last_unstale":"2023-06-23T18:03:42.921568+0000","last_undegraded":"2023-06-23T18:03:42.921568+0000","last_fullsized":"2023-06-23T18:03:42.921568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921045+0000","last_change":"2023-06-23T18:03:41.785038+0000","last_active":"2023-06-23T18:03:42.921045+0000","last_peered":"2023-06-23T18:03:42.921045+0000","last_clean":"2023-06-23T18:03:42.921045+0000","last_became_active":"2023-06-23T18:03:41.784124+0000","last_became_peered":"2023-06-23T18:03:41.784124+0000","last_unstale":"2023-06-23T18:03:42.921045+0000","last_undegraded":"2023-06-23T18:03:42.921045+0000","last_fullsized":"2023-06-23T18:03:42.921045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921205+0000","last_change":"2023-06-23T18:03:37.381450+0000","last_active":"2023-06-23T18:03:42.921205+0000","last_peered":"2023-06-23T18:03:42.921205+0000","last_clean":"2023-06-23T18:03:42.921205+0000","last_became_active":"2023-06-23T18:03:37.381137+0000","last_became_peered":"2023-06-23T18:03:37.381137+0000","last_unstale":"2023-06-23T18:03:42.921205+0000","last_undegraded":"2023-06-23T18:03:42.921205+0000","last_fullsized":"2023-06-23T18:03:42.921205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:04.690816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922338+0000","last_change":"2023-06-23T18:03:37.752538+0000","last_active":"2023-06-23T18:03:42.922338+0000","last_peered":"2023-06-23T18:03:42.922338+0000","last_clean":"2023-06-23T18:03:42.922338+0000","last_became_active":"2023-06-23T18:03:37.752276+0000","last_became_peered":"2023-06-23T18:03:37.752276+0000","last_unstale":"2023-06-23T18:03:42.922338+0000","last_undegraded":"2023-06-23T18:03:42.922338+0000","last_fullsized":"2023-06-23T18:03:42.922338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:23.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921015+0000","last_change":"2023-06-23T18:03:37.376067+0000","last_active":"2023-06-23T18:03:42.921015+0000","last_peered":"2023-06-23T18:03:42.921015+0000","last_clean":"2023-06-23T18:03:42.921015+0000","last_became_active":"2023-06-23T18:03:37.375582+0000","last_became_peered":"2023-06-23T18:03:37.375582+0000","last_unstale":"2023-06-23T18:03:42.921015+0000","last_undegraded":"2023-06-23T18:03:42.921015+0000","last_fullsized":"2023-06-23T18:03:42.921015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924004+0000","last_change":"2023-06-23T18:03:37.770794+0000","last_active":"2023-06-23T18:03:42.924004+0000","last_peered":"2023-06-23T18:03:42.924004+0000","last_clean":"2023-06-23T18:03:42.924004+0000","last_became_active":"2023-06-23T18:03:37.770508+0000","last_became_peered":"2023-06-23T18:03:37.770508+0000","last_unstale":"2023-06-23T18:03:42.924004+0000","last_undegraded":"2023-06-23T18:03:42.924004+0000","last_fullsized":"2023-06-23T18:03:42.924004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:27.966872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919085+0000","last_change":"2023-06-23T18:03:41.784964+0000","last_active":"2023-06-23T18:03:42.919085+0000","last_peered":"2023-06-23T18:03:42.919085+0000","last_clean":"2023-06-23T18:03:42.919085+0000","last_became_active":"2023-06-23T18:03:41.783875+0000","last_became_peered":"2023-06-23T18:03:41.783875+0000","last_unstale":"2023-06-23T18:03:42.919085+0000","last_undegraded":"2023-06-23T18:03:42.919085+0000","last_fullsized":"2023-06-23T18:03:42.919085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:03.189212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921280+0000","last_change":"2023-06-23T18:03:37.381471+0000","last_active":"2023-06-23T18:03:42.921280+0000","last_peered":"2023-06-23T18:03:42.921280+0000","last_clean":"2023-06-23T18:03:42.921280+0000","last_became_active":"2023-06-23T18:03:37.381080+0000","last_became_peered":"2023-06-23T18:03:37.381080+0000","last_unstale":"2023-06-23T18:03:42.921280+0000","last_undegraded":"2023-06-23T18:03:42.921280+0000","last_fullsized":"2023-06-23T18:03:42.921280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:36.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919682+0000","last_change":"2023-06-23T18:03:37.770191+0000","last_active":"2023-06-23T18:03:42.919682+0000","last_peered":"2023-06-23T18:03:42.919682+0000","last_clean":"2023-06-23T18:03:42.919682+0000","last_became_active":"2023-06-23T18:03:37.769918+0000","last_became_peered":"2023-06-23T18:03:37.769918+0000","last_unstale":"2023-06-23T18:03:42.919682+0000","last_undegraded":"2023-06-23T18:03:42.919682+0000","last_fullsized":"2023-06-23T18:03:42.919682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:36.439982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924483+0000","last_change":"2023-06-23T18:03:41.789295+0000","last_active":"2023-06-23T18:03:42.924483+0000","last_peered":"2023-06-23T18:03:42.924483+0000","last_clean":"2023-06-23T18:03:42.924483+0000","last_became_active":"2023-06-23T18:03:41.788890+0000","last_became_peered":"2023-06-23T18:03:41.788890+0000","last_unstale":"2023-06-23T18:03:42.924483+0000","last_undegraded":"2023-06-23T18:03:42.924483+0000","last_fullsized":"2023-06-23T18:03:42.924483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:36.106369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918865+0000","last_change":"2023-06-23T18:03:37.759952+0000","last_active":"2023-06-23T18:03:42.918865+0000","last_peered":"2023-06-23T18:03:42.918865+0000","last_clean":"2023-06-23T18:03:42.918865+0000","last_became_active":"2023-06-23T18:03:37.759685+0000","last_became_peered":"2023-06-23T18:03:37.759685+0000","last_unstale":"2023-06-23T18:03:42.918865+0000","last_undegraded":"2023-06-23T18:03:42.918865+0000","last_fullsized":"2023-06-23T18:03:42.918865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:01:11.356694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916611+0000","last_change":"2023-06-23T18:03:37.761169+0000","last_active":"2023-06-23T18:03:42.916611+0000","last_peered":"2023-06-23T18:03:42.916611+0000","last_clean":"2023-06-23T18:03:42.916611+0000","last_became_active":"2023-06-23T18:03:37.760915+0000","last_became_peered":"2023-06-23T18:03:37.760915+0000","last_unstale":"2023-06-23T18:03:42.916611+0000","last_undegraded":"2023-06-23T18:03:42.916611+0000","last_fullsized":"2023-06-23T18:03:42.916611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:15:41.693241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157660+0000","last_change":"2023-06-23T18:03:37.756608+0000","last_active":"2023-06-23T18:03:42.157660+0000","last_peered":"2023-06-23T18:03:42.157660+0000","last_clean":"2023-06-23T18:03:42.157660+0000","last_became_active":"2023-06-23T18:03:37.756267+0000","last_became_peered":"2023-06-23T18:03:37.756267+0000","last_unstale":"2023-06-23T18:03:42.157660+0000","last_undegraded":"2023-06-23T18:03:42.157660+0000","last_fullsized":"2023-06-23T18:03:42.157660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:39:56.896505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917984+0000","last_change":"2023-06-23T18:03:37.768762+0000","last_active":"2023-06-23T18:03:42.917984+0000","last_peered":"2023-06-23T18:03:42.917984+0000","last_clean":"2023-06-23T18:03:42.917984+0000","last_became_active":"2023-06-23T18:03:37.768493+0000","last_became_peered":"2023-06-23T18:03:37.768493+0000","last_unstale":"2023-06-23T18:03:42.917984+0000","last_undegraded":"2023-06-23T18:03:42.917984+0000","last_fullsized":"2023-06-23T18:03:42.917984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:47.275516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918340+0000","last_change":"2023-06-23T18:03:37.770023+0000","last_active":"2023-06-23T18:03:42.918340+0000","last_peered":"2023-06-23T18:03:42.918340+0000","last_clean":"2023-06-23T18:03:42.918340+0000","last_became_active":"2023-06-23T18:03:37.769776+0000","last_became_peered":"2023-06-23T18:03:37.769776+0000","last_unstale":"2023-06-23T18:03:42.918340+0000","last_undegraded":"2023-06-23T18:03:42.918340+0000","last_fullsized":"2023-06-23T18:03:42.918340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923845+0000","last_change":"2023-06-23T18:03:37.772405+0000","last_active":"2023-06-23T18:03:42.923845+0000","last_peered":"2023-06-23T18:03:42.923845+0000","last_clean":"2023-06-23T18:03:42.923845+0000","last_became_active":"2023-06-23T18:03:37.772086+0000","last_became_peered":"2023-06-23T18:03:37.772086+0000","last_unstale":"2023-06-23T18:03:42.923845+0000","last_undegraded":"2023-06-23T18:03:42.923845+0000","last_fullsized":"2023-06-23T18:03:42.923845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:49.758628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919917+0000","last_change":"2023-06-23T18:03:37.768948+0000","last_active":"2023-06-23T18:03:42.919917+0000","last_peered":"2023-06-23T18:03:42.919917+0000","last_clean":"2023-06-23T18:03:42.919917+0000","last_became_active":"2023-06-23T18:03:37.768355+0000","last_became_peered":"2023-06-23T18:03:37.768355+0000","last_unstale":"2023-06-23T18:03:42.919917+0000","last_undegraded":"2023-06-23T18:03:42.919917+0000","last_fullsized":"2023-06-23T18:03:42.919917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919527+0000","last_change":"2023-06-23T18:03:37.764486+0000","last_active":"2023-06-23T18:03:42.919527+0000","last_peered":"2023-06-23T18:03:42.919527+0000","last_clean":"2023-06-23T18:03:42.919527+0000","last_became_active":"2023-06-23T18:03:37.755419+0000","last_became_peered":"2023-06-23T18:03:37.755419+0000","last_unstale":"2023-06-23T18:03:42.919527+0000","last_undegraded":"2023-06-23T18:03:42.919527+0000","last_fullsized":"2023-06-23T18:03:42.919527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921001+0000","last_change":"2023-06-23T18:03:37.770648+0000","last_active":"2023-06-23T18:03:42.921001+0000","last_peered":"2023-06-23T18:03:42.921001+0000","last_clean":"2023-06-23T18:03:42.921001+0000","last_became_active":"2023-06-23T18:03:37.770271+0000","last_became_peered":"2023-06-23T18:03:37.770271+0000","last_unstale":"2023-06-23T18:03:42.921001+0000","last_undegraded":"2023-06-23T18:03:42.921001+0000","last_fullsized":"2023-06-23T18:03:42.921001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916514+0000","last_change":"2023-06-23T18:03:37.767363+0000","last_active":"2023-06-23T18:03:42.916514+0000","last_peered":"2023-06-23T18:03:42.916514+0000","last_clean":"2023-06-23T18:03:42.916514+0000","last_became_active":"2023-06-23T18:03:37.766918+0000","last_became_peered":"2023-06-23T18:03:37.766918+0000","last_unstale":"2023-06-23T18:03:42.916514+0000","last_undegraded":"2023-06-23T18:03:42.916514+0000","last_fullsized":"2023-06-23T18:03:42.916514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:31.121336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920725+0000","last_change":"2023-06-23T18:03:37.757526+0000","last_active":"2023-06-23T18:03:42.920725+0000","last_peered":"2023-06-23T18:03:42.920725+0000","last_clean":"2023-06-23T18:03:42.920725+0000","last_became_active":"2023-06-23T18:03:37.757223+0000","last_became_peered":"2023-06-23T18:03:37.757223+0000","last_unstale":"2023-06-23T18:03:42.920725+0000","last_undegraded":"2023-06-23T18:03:42.920725+0000","last_fullsized":"2023-06-23T18:03:42.920725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921071+0000","last_change":"2023-06-23T18:03:37.770932+0000","last_active":"2023-06-23T18:03:42.921071+0000","last_peered":"2023-06-23T18:03:42.921071+0000","last_clean":"2023-06-23T18:03:42.921071+0000","last_became_active":"2023-06-23T18:03:37.770633+0000","last_became_peered":"2023-06-23T18:03:37.770633+0000","last_unstale":"2023-06-23T18:03:42.921071+0000","last_undegraded":"2023-06-23T18:03:42.921071+0000","last_fullsized":"2023-06-23T18:03:42.921071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158738+0000","last_change":"2023-06-23T18:03:37.757510+0000","last_active":"2023-06-23T18:03:42.158738+0000","last_peered":"2023-06-23T18:03:42.158738+0000","last_clean":"2023-06-23T18:03:42.158738+0000","last_became_active":"2023-06-23T18:03:37.757203+0000","last_became_peered":"2023-06-23T18:03:37.757203+0000","last_unstale":"2023-06-23T18:03:42.158738+0000","last_undegraded":"2023-06-23T18:03:42.158738+0000","last_fullsized":"2023-06-23T18:03:42.158738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:25:37.629512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153090+0000","last_change":"2023-06-23T18:03:37.760931+0000","last_active":"2023-06-23T18:03:42.153090+0000","last_peered":"2023-06-23T18:03:42.153090+0000","last_clean":"2023-06-23T18:03:42.153090+0000","last_became_active":"2023-06-23T18:03:37.760672+0000","last_became_peered":"2023-06-23T18:03:37.760672+0000","last_unstale":"2023-06-23T18:03:42.153090+0000","last_undegraded":"2023-06-23T18:03:42.153090+0000","last_fullsized":"2023-06-23T18:03:42.153090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:35.350379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158558+0000","last_change":"2023-06-23T18:03:37.761315+0000","last_active":"2023-06-23T18:03:42.158558+0000","last_peered":"2023-06-23T18:03:42.158558+0000","last_clean":"2023-06-23T18:03:42.158558+0000","last_became_active":"2023-06-23T18:03:37.760975+0000","last_became_peered":"2023-06-23T18:03:37.760975+0000","last_unstale":"2023-06-23T18:03:42.158558+0000","last_undegraded":"2023-06-23T18:03:42.158558+0000","last_fullsized":"2023-06-23T18:03:42.158558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:04.033803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922192+0000","last_change":"2023-06-23T18:03:37.754659+0000","last_active":"2023-06-23T18:03:42.922192+0000","last_peered":"2023-06-23T18:03:42.922192+0000","last_clean":"2023-06-23T18:03:42.922192+0000","last_became_active":"2023-06-23T18:03:37.751277+0000","last_became_peered":"2023-06-23T18:03:37.751277+0000","last_unstale":"2023-06-23T18:03:42.922192+0000","last_undegraded":"2023-06-23T18:03:42.922192+0000","last_fullsized":"2023-06-23T18:03:42.922192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156785+0000","last_change":"2023-06-23T18:03:37.752494+0000","last_active":"2023-06-23T18:03:42.156785+0000","last_peered":"2023-06-23T18:03:42.156785+0000","last_clean":"2023-06-23T18:03:42.156785+0000","last_became_active":"2023-06-23T18:03:37.752149+0000","last_became_peered":"2023-06-23T18:03:37.752149+0000","last_unstale":"2023-06-23T18:03:42.156785+0000","last_undegraded":"2023-06-23T18:03:42.156785+0000","last_fullsized":"2023-06-23T18:03:42.156785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:37.654382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921056+0000","last_change":"2023-06-23T18:03:37.775997+0000","last_active":"2023-06-23T18:03:42.921056+0000","last_peered":"2023-06-23T18:03:42.921056+0000","last_clean":"2023-06-23T18:03:42.921056+0000","last_became_active":"2023-06-23T18:03:37.775685+0000","last_became_peered":"2023-06-23T18:03:37.775685+0000","last_unstale":"2023-06-23T18:03:42.921056+0000","last_undegraded":"2023-06-23T18:03:42.921056+0000","last_fullsized":"2023-06-23T18:03:42.921056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923426+0000","last_change":"2023-06-23T18:03:37.763367+0000","last_active":"2023-06-23T18:03:42.923426+0000","last_peered":"2023-06-23T18:03:42.923426+0000","last_clean":"2023-06-23T18:03:42.923426+0000","last_became_active":"2023-06-23T18:03:37.763035+0000","last_became_peered":"2023-06-23T18:03:37.763035+0000","last_unstale":"2023-06-23T18:03:42.923426+0000","last_undegraded":"2023-06-23T18:03:42.923426+0000","last_fullsized":"2023-06-23T18:03:42.923426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152175+0000","last_change":"2023-06-23T18:03:37.758554+0000","last_active":"2023-06-23T18:03:42.152175+0000","last_peered":"2023-06-23T18:03:42.152175+0000","last_clean":"2023-06-23T18:03:42.152175+0000","last_became_active":"2023-06-23T18:03:37.758307+0000","last_became_peered":"2023-06-23T18:03:37.758307+0000","last_unstale":"2023-06-23T18:03:42.152175+0000","last_undegraded":"2023-06-23T18:03:42.152175+0000","last_fullsized":"2023-06-23T18:03:42.152175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:23:09.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152731+0000","last_change":"2023-06-23T18:03:37.760705+0000","last_active":"2023-06-23T18:03:42.152731+0000","last_peered":"2023-06-23T18:03:42.152731+0000","last_clean":"2023-06-23T18:03:42.152731+0000","last_became_active":"2023-06-23T18:03:37.759846+0000","last_became_peered":"2023-06-23T18:03:37.759846+0000","last_unstale":"2023-06-23T18:03:42.152731+0000","last_undegraded":"2023-06-23T18:03:42.152731+0000","last_fullsized":"2023-06-23T18:03:42.152731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:26.801591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158644+0000","last_change":"2023-06-23T18:03:37.757927+0000","last_active":"2023-06-23T18:03:42.158644+0000","last_peered":"2023-06-23T18:03:42.158644+0000","last_clean":"2023-06-23T18:03:42.158644+0000","last_became_active":"2023-06-23T18:03:37.757635+0000","last_became_peered":"2023-06-23T18:03:37.757635+0000","last_unstale":"2023-06-23T18:03:42.158644+0000","last_undegraded":"2023-06-23T18:03:42.158644+0000","last_fullsized":"2023-06-23T18:03:42.158644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:26.427732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923450+0000","last_change":"2023-06-23T18:03:37.768282+0000","last_active":"2023-06-23T18:03:42.923450+0000","last_peered":"2023-06-23T18:03:42.923450+0000","last_clean":"2023-06-23T18:03:42.923450+0000","last_became_active":"2023-06-23T18:03:37.768038+0000","last_became_peered":"2023-06-23T18:03:37.768038+0000","last_unstale":"2023-06-23T18:03:42.923450+0000","last_undegraded":"2023-06-23T18:03:42.923450+0000","last_fullsized":"2023-06-23T18:03:42.923450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922018+0000","last_change":"2023-06-23T18:03:37.752981+0000","last_active":"2023-06-23T18:03:42.922018+0000","last_peered":"2023-06-23T18:03:42.922018+0000","last_clean":"2023-06-23T18:03:42.922018+0000","last_became_active":"2023-06-23T18:03:37.752291+0000","last_became_peered":"2023-06-23T18:03:37.752291+0000","last_unstale":"2023-06-23T18:03:42.922018+0000","last_undegraded":"2023-06-23T18:03:42.922018+0000","last_fullsized":"2023-06-23T18:03:42.922018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919440+0000","last_change":"2023-06-23T18:03:37.756558+0000","last_active":"2023-06-23T18:03:42.919440+0000","last_peered":"2023-06-23T18:03:42.919440+0000","last_clean":"2023-06-23T18:03:42.919440+0000","last_became_active":"2023-06-23T18:03:37.756044+0000","last_became_peered":"2023-06-23T18:03:37.756044+0000","last_unstale":"2023-06-23T18:03:42.919440+0000","last_undegraded":"2023-06-23T18:03:42.919440+0000","last_fullsized":"2023-06-23T18:03:42.919440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920901+0000","last_change":"2023-06-23T18:03:37.775992+0000","last_active":"2023-06-23T18:03:42.920901+0000","last_peered":"2023-06-23T18:03:42.920901+0000","last_clean":"2023-06-23T18:03:42.920901+0000","last_became_active":"2023-06-23T18:03:37.775667+0000","last_became_peered":"2023-06-23T18:03:37.775667+0000","last_unstale":"2023-06-23T18:03:42.920901+0000","last_undegraded":"2023-06-23T18:03:42.920901+0000","last_fullsized":"2023-06-23T18:03:42.920901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157021+0000","last_change":"2023-06-23T18:03:37.752811+0000","last_active":"2023-06-23T18:03:42.157021+0000","last_peered":"2023-06-23T18:03:42.157021+0000","last_clean":"2023-06-23T18:03:42.157021+0000","last_became_active":"2023-06-23T18:03:37.752518+0000","last_became_peered":"2023-06-23T18:03:37.752518+0000","last_unstale":"2023-06-23T18:03:42.157021+0000","last_undegraded":"2023-06-23T18:03:42.157021+0000","last_fullsized":"2023-06-23T18:03:42.157021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:26.870651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920914+0000","last_change":"2023-06-23T18:03:37.757769+0000","last_active":"2023-06-23T18:03:42.920914+0000","last_peered":"2023-06-23T18:03:42.920914+0000","last_clean":"2023-06-23T18:03:42.920914+0000","last_became_active":"2023-06-23T18:03:37.757520+0000","last_became_peered":"2023-06-23T18:03:37.757520+0000","last_unstale":"2023-06-23T18:03:42.920914+0000","last_undegraded":"2023-06-23T18:03:42.920914+0000","last_fullsized":"2023-06-23T18:03:42.920914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919630+0000","last_change":"2023-06-23T18:03:37.755865+0000","last_active":"2023-06-23T18:03:42.919630+0000","last_peered":"2023-06-23T18:03:42.919630+0000","last_clean":"2023-06-23T18:03:42.919630+0000","last_became_active":"2023-06-23T18:03:37.755153+0000","last_became_peered":"2023-06-23T18:03:37.755153+0000","last_unstale":"2023-06-23T18:03:42.919630+0000","last_undegraded":"2023-06-23T18:03:42.919630+0000","last_fullsized":"2023-06-23T18:03:42.919630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:42:12.603650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158475+0000","last_change":"2023-06-23T18:03:37.754864+0000","last_active":"2023-06-23T18:03:42.158475+0000","last_peered":"2023-06-23T18:03:42.158475+0000","last_clean":"2023-06-23T18:03:42.158475+0000","last_became_active":"2023-06-23T18:03:37.754386+0000","last_became_peered":"2023-06-23T18:03:37.754386+0000","last_unstale":"2023-06-23T18:03:42.158475+0000","last_undegraded":"2023-06-23T18:03:42.158475+0000","last_fullsized":"2023-06-23T18:03:42.158475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:30:06.917401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922699+0000","last_change":"2023-06-23T18:03:37.757620+0000","last_active":"2023-06-23T18:03:42.922699+0000","last_peered":"2023-06-23T18:03:42.922699+0000","last_clean":"2023-06-23T18:03:42.922699+0000","last_became_active":"2023-06-23T18:03:37.757346+0000","last_became_peered":"2023-06-23T18:03:37.757346+0000","last_unstale":"2023-06-23T18:03:42.922699+0000","last_undegraded":"2023-06-23T18:03:42.922699+0000","last_fullsized":"2023-06-23T18:03:42.922699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:59.832808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152893+0000","last_change":"2023-06-23T18:03:37.759638+0000","last_active":"2023-06-23T18:03:42.152893+0000","last_peered":"2023-06-23T18:03:42.152893+0000","last_clean":"2023-06-23T18:03:42.152893+0000","last_became_active":"2023-06-23T18:03:37.759380+0000","last_became_peered":"2023-06-23T18:03:37.759380+0000","last_unstale":"2023-06-23T18:03:42.152893+0000","last_undegraded":"2023-06-23T18:03:42.152893+0000","last_fullsized":"2023-06-23T18:03:42.152893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:44.739308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31300,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340540,"statfs":{"total":96636764160,"available":96604712960,"internally_reserved":0,"allocated":258048,"data_stored":104812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":851968,"data_stored":695672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:04:01.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:04:01.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 Processor -- start 2023-06-23T18:04:01.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 -- start start 2023-06-23T18:04:01.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c04159c30 0x7f6c040afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040b0420 con 0x7f6c040affd0 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0406d0d0 con 0x7f6c04071450 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c03fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0406d560 con 0x7f6c04159c30 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c027fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6c040affd0 0x7f6c04070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37104/0 (socket says 172.21.15.22:37104) 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c027fc700 1 -- 172.21.15.22:0/4136498864 learned_addr learned my addr 172.21.15.22:0/4136498864 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:01.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163071159 0 0) 0x7f6bf4001110 con 0x7f6c040affd0 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bec0035d0 con 0x7f6c040affd0 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c02ffd700 1 --2- 172.21.15.22:0/4136498864 >> v2:172.21.15.169:3300/0 conn(0x7f6c04159c30 0x7f6c040afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3405894170 0 0) 0x7f6bec0035d0 con 0x7f6c040affd0 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 >> v2:172.21.15.169:3300/0 conn(0x7f6c04159c30 msgr2=0x7f6c040afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 --2- 172.21.15.22:0/4136498864 >> v2:172.21.15.169:3300/0 conn(0x7f6c04159c30 0x7f6c040afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 >> v1:172.21.15.169:6789/0 conn(0x7f6c04071450 legacy=0x7f6c040cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0406c500 con 0x7f6c040affd0 2023-06-23T18:04:01.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f6bf4001800 con 0x7f6c040affd0 2023-06-23T18:04:01.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 >> v1:172.21.15.22:6789/0 conn(0x7f6c040affd0 legacy=0x7f6c04070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bec003a20 0x7f6bec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec004870 0x7f6bec004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 0x7f6bec00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0406d560 con 0x7f6bec005340 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec00c200 con 0x7f6bec004870 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec00c3c0 con 0x7f6bec003a20 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.859+0000 7f6c037fe700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 0x7f6bec00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c027fc700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec004870 0x7f6bec004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c02ffd700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bec003a20 0x7f6bec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 -- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bec003a20 msgr2=0x7f6bec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:01.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bec003a20 0x7f6bec003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 -- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec004870 msgr2=0x7f6bec004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec004870 0x7f6bec004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 -- 172.21.15.22:0/4136498864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0035d0 con 0x7f6bec005340 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c027fc700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec004870 0x7f6bec004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 -- 172.21.15.22:0/4136498864 reap_dead start 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c037fe700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 0x7f6bec00ba30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00c190 tx=0x7f6bec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2122a74acfa02052 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc001db0 con 0x7f6bec005340 2023-06-23T18:04:01.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00ccf0 con 0x7f6bec005340 2023-06-23T18:04:01.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c01ffb700 1 -- 172.21.15.22:0/4136498864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc003c60 con 0x7f6bec005340 2023-06-23T18:04:01.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 msgr2=0x7f6bec00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 0x7f6bec00ba30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00c190 tx=0x7f6bec0058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- 172.21.15.22:0/4136498864 shutdown_connections 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 --2- 172.21.15.22:0/4136498864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec005340 0x7f6bec00ba30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- 172.21.15.22:0/4136498864 >> 172.21.15.22:0/4136498864 conn(0x7f6c0406a5f0 msgr2=0x7f6c040caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- 172.21.15.22:0/4136498864 shutdown_connections 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- 172.21.15.22:0/4136498864 wait complete. 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 Processor -- start 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.863+0000 7f6c03fff700 1 -- start start 2023-06-23T18:04:01.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec005340 0x7f6c041f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c040c7ca0 0x7f6c0415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0415b350 con 0x7f6c041f3620 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0415a2b0 con 0x7f6bec005340 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0415a590 con 0x7f6c040c7ca0 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43824/0 (socket says 172.21.15.22:43824) 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 -- 172.21.15.22:0/3345842017 learned_addr learned my addr 172.21.15.22:0/3345842017 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c02ffd700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec005340 0x7f6c041f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c037fe700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c040c7ca0 0x7f6c0415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 -- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c040c7ca0 msgr2=0x7f6c0415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c040c7ca0 0x7f6c0415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 -- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec005340 msgr2=0x7f6c041f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:01.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec005340 0x7f6c041f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 -- 172.21.15.22:0/3345842017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0415a870 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c027fc700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb20 tx=0x7f6bf400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400c5d0 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf400c730 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400cac0 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0415ab50 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.867+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0415c660 con 0x7f6c041f3620 2023-06-23T18:04:01.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c040c8070 con 0x7f6c041f3620 2023-06-23T18:04:01.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6bf400cc20 con 0x7f6c041f3620 2023-06-23T18:04:01.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6beb7fe700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 0x7f6bf00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:01.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6bf4083b00 con 0x7f6c041f3620 2023-06-23T18:04:01.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6c02ffd700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 0x7f6bf00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:01.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.871+0000 7f6c02ffd700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 0x7f6bf00687a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:01.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:01.879+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6bf4050c10 con 0x7f6c041f3620 2023-06-23T18:04:02.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.043+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c0415a700 con 0x7f6bf00662f0 2023-06-23T18:04:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.059+0000 7f6beb7fe700 1 -- 172.21.15.22:0/3345842017 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367266 (secure 0 0 0) 0x7f6c0415a700 con 0x7f6bf00662f0 2023-06-23T18:04:02.079 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 msgr2=0x7f6bf00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 0x7f6bf00687a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8008040 comp rx=0 tx=0).stop 2023-06-23T18:04:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 msgr2=0x7f6c040c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb20 tx=0x7f6bf400ee30 comp rx=0 tx=0).stop 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 shutdown_connections 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c040c7ca0 0x7f6c0415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c041f3620 0x7f6c040c7760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6bf00662f0 0x7f6bf00687a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 --2- 172.21.15.22:0/3345842017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bec005340 0x7f6c041f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 >> 172.21.15.22:0/3345842017 conn(0x7f6c0406a5f0 msgr2=0x7f6c040aada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 shutdown_connections 2023-06-23T18:04:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.067+0000 7f6c03fff700 1 -- 172.21.15.22:0/3345842017 wait complete. 2023-06-23T18:04:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:04:02.097 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-06-23T18:04:01.615485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235208,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":3284992,"data_stored":2027694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004868"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915552+0000","last_change":"2023-06-23T18:03:41.789565+0000","last_active":"2023-06-23T18:03:42.915552+0000","last_peered":"2023-06-23T18:03:42.915552+0000","last_clean":"2023-06-23T18:03:42.915552+0000","last_became_active":"2023-06-23T18:03:41.787653+0000","last_became_peered":"2023-06-23T18:03:41.787653+0000","last_unstale":"2023-06-23T18:03:42.915552+0000","last_undegraded":"2023-06-23T18:03:42.915552+0000","last_fullsized":"2023-06-23T18:03:42.915552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922374+0000","last_change":"2023-06-23T18:03:37.377461+0000","last_active":"2023-06-23T18:03:42.922374+0000","last_peered":"2023-06-23T18:03:42.922374+0000","last_clean":"2023-06-23T18:03:42.922374+0000","last_became_active":"2023-06-23T18:03:37.377017+0000","last_became_peered":"2023-06-23T18:03:37.377017+0000","last_unstale":"2023-06-23T18:03:42.922374+0000","last_undegraded":"2023-06-23T18:03:42.922374+0000","last_fullsized":"2023-06-23T18:03:42.922374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157012+0000","last_change":"2023-06-23T18:03:37.756400+0000","last_active":"2023-06-23T18:03:42.157012+0000","last_peered":"2023-06-23T18:03:42.157012+0000","last_clean":"2023-06-23T18:03:42.157012+0000","last_became_active":"2023-06-23T18:03:37.755870+0000","last_became_peered":"2023-06-23T18:03:37.755870+0000","last_unstale":"2023-06-23T18:03:42.157012+0000","last_undegraded":"2023-06-23T18:03:42.157012+0000","last_fullsized":"2023-06-23T18:03:42.157012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:35:48.712508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157286+0000","last_change":"2023-06-23T18:03:41.784738+0000","last_active":"2023-06-23T18:03:42.157286+0000","last_peered":"2023-06-23T18:03:42.157286+0000","last_clean":"2023-06-23T18:03:42.157286+0000","last_became_active":"2023-06-23T18:03:41.784348+0000","last_became_peered":"2023-06-23T18:03:41.784348+0000","last_unstale":"2023-06-23T18:03:42.157286+0000","last_undegraded":"2023-06-23T18:03:42.157286+0000","last_fullsized":"2023-06-23T18:03:42.157286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:06:40.211756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920797+0000","last_change":"2023-06-23T18:03:37.364428+0000","last_active":"2023-06-23T18:03:42.920797+0000","last_peered":"2023-06-23T18:03:42.920797+0000","last_clean":"2023-06-23T18:03:42.920797+0000","last_became_active":"2023-06-23T18:03:37.364114+0000","last_became_peered":"2023-06-23T18:03:37.364114+0000","last_unstale":"2023-06-23T18:03:42.920797+0000","last_undegraded":"2023-06-23T18:03:42.920797+0000","last_fullsized":"2023-06-23T18:03:42.920797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:53.340210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917007+0000","last_change":"2023-06-23T18:03:37.761252+0000","last_active":"2023-06-23T18:03:42.917007+0000","last_peered":"2023-06-23T18:03:42.917007+0000","last_clean":"2023-06-23T18:03:42.917007+0000","last_became_active":"2023-06-23T18:03:37.760991+0000","last_became_peered":"2023-06-23T18:03:37.760991+0000","last_unstale":"2023-06-23T18:03:42.917007+0000","last_undegraded":"2023-06-23T18:03:42.917007+0000","last_fullsized":"2023-06-23T18:03:42.917007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:26.556145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922896+0000","last_change":"2023-06-23T18:03:41.789561+0000","last_active":"2023-06-23T18:03:42.922896+0000","last_peered":"2023-06-23T18:03:42.922896+0000","last_clean":"2023-06-23T18:03:42.922896+0000","last_became_active":"2023-06-23T18:03:41.788768+0000","last_became_peered":"2023-06-23T18:03:41.788768+0000","last_unstale":"2023-06-23T18:03:42.922896+0000","last_undegraded":"2023-06-23T18:03:42.922896+0000","last_fullsized":"2023-06-23T18:03:42.922896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924155+0000","last_change":"2023-06-23T18:03:37.377124+0000","last_active":"2023-06-23T18:03:42.924155+0000","last_peered":"2023-06-23T18:03:42.924155+0000","last_clean":"2023-06-23T18:03:42.924155+0000","last_became_active":"2023-06-23T18:03:37.376829+0000","last_became_peered":"2023-06-23T18:03:37.376829+0000","last_unstale":"2023-06-23T18:03:42.924155+0000","last_undegraded":"2023-06-23T18:03:42.924155+0000","last_fullsized":"2023-06-23T18:03:42.924155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157618+0000","last_change":"2023-06-23T18:03:37.754592+0000","last_active":"2023-06-23T18:03:42.157618+0000","last_peered":"2023-06-23T18:03:42.157618+0000","last_clean":"2023-06-23T18:03:42.157618+0000","last_became_active":"2023-06-23T18:03:37.754279+0000","last_became_peered":"2023-06-23T18:03:37.754279+0000","last_unstale":"2023-06-23T18:03:42.157618+0000","last_undegraded":"2023-06-23T18:03:42.157618+0000","last_fullsized":"2023-06-23T18:03:42.157618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:05:29.058510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920916+0000","last_change":"2023-06-23T18:03:41.787327+0000","last_active":"2023-06-23T18:03:42.920916+0000","last_peered":"2023-06-23T18:03:42.920916+0000","last_clean":"2023-06-23T18:03:42.920916+0000","last_became_active":"2023-06-23T18:03:41.786664+0000","last_became_peered":"2023-06-23T18:03:41.786664+0000","last_unstale":"2023-06-23T18:03:42.920916+0000","last_undegraded":"2023-06-23T18:03:42.920916+0000","last_fullsized":"2023-06-23T18:03:42.920916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922619+0000","last_change":"2023-06-23T18:03:37.377468+0000","last_active":"2023-06-23T18:03:42.922619+0000","last_peered":"2023-06-23T18:03:42.922619+0000","last_clean":"2023-06-23T18:03:42.922619+0000","last_became_active":"2023-06-23T18:03:37.376653+0000","last_became_peered":"2023-06-23T18:03:37.376653+0000","last_unstale":"2023-06-23T18:03:42.922619+0000","last_undegraded":"2023-06-23T18:03:42.922619+0000","last_fullsized":"2023-06-23T18:03:42.922619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157926+0000","last_change":"2023-06-23T18:03:37.761926+0000","last_active":"2023-06-23T18:03:42.157926+0000","last_peered":"2023-06-23T18:03:42.157926+0000","last_clean":"2023-06-23T18:03:42.157926+0000","last_became_active":"2023-06-23T18:03:37.761621+0000","last_became_peered":"2023-06-23T18:03:37.761621+0000","last_unstale":"2023-06-23T18:03:42.157926+0000","last_undegraded":"2023-06-23T18:03:42.157926+0000","last_fullsized":"2023-06-23T18:03:42.157926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:20.322939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918838+0000","last_change":"2023-06-23T18:03:41.782917+0000","last_active":"2023-06-23T18:03:42.918838+0000","last_peered":"2023-06-23T18:03:42.918838+0000","last_clean":"2023-06-23T18:03:42.918838+0000","last_became_active":"2023-06-23T18:03:41.782516+0000","last_became_peered":"2023-06-23T18:03:41.782516+0000","last_unstale":"2023-06-23T18:03:42.918838+0000","last_undegraded":"2023-06-23T18:03:42.918838+0000","last_fullsized":"2023-06-23T18:03:42.918838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920996+0000","last_change":"2023-06-23T18:03:37.379511+0000","last_active":"2023-06-23T18:03:42.920996+0000","last_peered":"2023-06-23T18:03:42.920996+0000","last_clean":"2023-06-23T18:03:42.920996+0000","last_became_active":"2023-06-23T18:03:37.379252+0000","last_became_peered":"2023-06-23T18:03:37.379252+0000","last_unstale":"2023-06-23T18:03:42.920996+0000","last_undegraded":"2023-06-23T18:03:42.920996+0000","last_fullsized":"2023-06-23T18:03:42.920996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:58.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153021+0000","last_change":"2023-06-23T18:03:37.768862+0000","last_active":"2023-06-23T18:03:42.153021+0000","last_peered":"2023-06-23T18:03:42.153021+0000","last_clean":"2023-06-23T18:03:42.153021+0000","last_became_active":"2023-06-23T18:03:37.768610+0000","last_became_peered":"2023-06-23T18:03:37.768610+0000","last_unstale":"2023-06-23T18:03:42.153021+0000","last_undegraded":"2023-06-23T18:03:42.153021+0000","last_fullsized":"2023-06-23T18:03:42.153021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:10:03.453386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158181+0000","last_change":"2023-06-23T18:03:41.781852+0000","last_active":"2023-06-23T18:03:42.158181+0000","last_peered":"2023-06-23T18:03:42.158181+0000","last_clean":"2023-06-23T18:03:42.158181+0000","last_became_active":"2023-06-23T18:03:41.781321+0000","last_became_peered":"2023-06-23T18:03:41.781321+0000","last_unstale":"2023-06-23T18:03:42.158181+0000","last_undegraded":"2023-06-23T18:03:42.158181+0000","last_fullsized":"2023-06-23T18:03:42.158181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:21:44.946317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157942+0000","last_change":"2023-06-23T18:03:37.372568+0000","last_active":"2023-06-23T18:03:42.157942+0000","last_peered":"2023-06-23T18:03:42.157942+0000","last_clean":"2023-06-23T18:03:42.157942+0000","last_became_active":"2023-06-23T18:03:37.372256+0000","last_became_peered":"2023-06-23T18:03:37.372256+0000","last_unstale":"2023-06-23T18:03:42.157942+0000","last_undegraded":"2023-06-23T18:03:42.157942+0000","last_fullsized":"2023-06-23T18:03:42.157942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:01:27.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919391+0000","last_change":"2023-06-23T18:03:37.761387+0000","last_active":"2023-06-23T18:03:42.919391+0000","last_peered":"2023-06-23T18:03:42.919391+0000","last_clean":"2023-06-23T18:03:42.919391+0000","last_became_active":"2023-06-23T18:03:37.758859+0000","last_became_peered":"2023-06-23T18:03:37.758859+0000","last_unstale":"2023-06-23T18:03:42.919391+0000","last_undegraded":"2023-06-23T18:03:42.919391+0000","last_fullsized":"2023-06-23T18:03:42.919391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919777+0000","last_change":"2023-06-23T18:03:41.782177+0000","last_active":"2023-06-23T18:03:42.919777+0000","last_peered":"2023-06-23T18:03:42.919777+0000","last_clean":"2023-06-23T18:03:42.919777+0000","last_became_active":"2023-06-23T18:03:41.781457+0000","last_became_peered":"2023-06-23T18:03:41.781457+0000","last_unstale":"2023-06-23T18:03:42.919777+0000","last_undegraded":"2023-06-23T18:03:42.919777+0000","last_fullsized":"2023-06-23T18:03:42.919777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:38:17.449084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916697+0000","last_change":"2023-06-23T18:03:37.367776+0000","last_active":"2023-06-23T18:03:42.916697+0000","last_peered":"2023-06-23T18:03:42.916697+0000","last_clean":"2023-06-23T18:03:42.916697+0000","last_became_active":"2023-06-23T18:03:37.367288+0000","last_became_peered":"2023-06-23T18:03:37.367288+0000","last_unstale":"2023-06-23T18:03:42.916697+0000","last_undegraded":"2023-06-23T18:03:42.916697+0000","last_fullsized":"2023-06-23T18:03:42.916697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:46.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922187+0000","last_change":"2023-06-23T18:03:37.769820+0000","last_active":"2023-06-23T18:03:42.922187+0000","last_peered":"2023-06-23T18:03:42.922187+0000","last_clean":"2023-06-23T18:03:42.922187+0000","last_became_active":"2023-06-23T18:03:37.769571+0000","last_became_peered":"2023-06-23T18:03:37.769571+0000","last_unstale":"2023-06-23T18:03:42.922187+0000","last_undegraded":"2023-06-23T18:03:42.922187+0000","last_fullsized":"2023-06-23T18:03:42.922187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921931+0000","last_change":"2023-06-23T18:03:41.783182+0000","last_active":"2023-06-23T18:03:42.921931+0000","last_peered":"2023-06-23T18:03:42.921931+0000","last_clean":"2023-06-23T18:03:42.921931+0000","last_became_active":"2023-06-23T18:03:41.782753+0000","last_became_peered":"2023-06-23T18:03:41.782753+0000","last_unstale":"2023-06-23T18:03:42.921931+0000","last_undegraded":"2023-06-23T18:03:42.921931+0000","last_fullsized":"2023-06-23T18:03:42.921931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916193+0000","last_change":"2023-06-23T18:03:37.378552+0000","last_active":"2023-06-23T18:03:42.916193+0000","last_peered":"2023-06-23T18:03:42.916193+0000","last_clean":"2023-06-23T18:03:42.916193+0000","last_became_active":"2023-06-23T18:03:37.378264+0000","last_became_peered":"2023-06-23T18:03:37.378264+0000","last_unstale":"2023-06-23T18:03:42.916193+0000","last_undegraded":"2023-06-23T18:03:42.916193+0000","last_fullsized":"2023-06-23T18:03:42.916193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:04:27.977965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920059+0000","last_change":"2023-06-23T18:03:37.755621+0000","last_active":"2023-06-23T18:03:42.920059+0000","last_peered":"2023-06-23T18:03:42.920059+0000","last_clean":"2023-06-23T18:03:42.920059+0000","last_became_active":"2023-06-23T18:03:37.755341+0000","last_became_peered":"2023-06-23T18:03:37.755341+0000","last_unstale":"2023-06-23T18:03:42.920059+0000","last_undegraded":"2023-06-23T18:03:42.920059+0000","last_fullsized":"2023-06-23T18:03:42.920059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:11:22.805985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915863+0000","last_change":"2023-06-23T18:03:37.758569+0000","last_active":"2023-06-23T18:03:42.915863+0000","last_peered":"2023-06-23T18:03:42.915863+0000","last_clean":"2023-06-23T18:03:42.915863+0000","last_became_active":"2023-06-23T18:03:37.758229+0000","last_became_peered":"2023-06-23T18:03:37.758229+0000","last_unstale":"2023-06-23T18:03:42.915863+0000","last_undegraded":"2023-06-23T18:03:42.915863+0000","last_fullsized":"2023-06-23T18:03:42.915863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:56:28.950360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922336+0000","last_change":"2023-06-23T18:03:37.377384+0000","last_active":"2023-06-23T18:03:42.922336+0000","last_peered":"2023-06-23T18:03:42.922336+0000","last_clean":"2023-06-23T18:03:42.922336+0000","last_became_active":"2023-06-23T18:03:37.376940+0000","last_became_peered":"2023-06-23T18:03:37.376940+0000","last_unstale":"2023-06-23T18:03:42.922336+0000","last_undegraded":"2023-06-23T18:03:42.922336+0000","last_fullsized":"2023-06-23T18:03:42.922336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919105+0000","last_change":"2023-06-23T18:03:41.787485+0000","last_active":"2023-06-23T18:03:42.919105+0000","last_peered":"2023-06-23T18:03:42.919105+0000","last_clean":"2023-06-23T18:03:42.919105+0000","last_became_active":"2023-06-23T18:03:41.787017+0000","last_became_peered":"2023-06-23T18:03:41.787017+0000","last_unstale":"2023-06-23T18:03:42.919105+0000","last_undegraded":"2023-06-23T18:03:42.919105+0000","last_fullsized":"2023-06-23T18:03:42.919105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:48:29.879084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916767+0000","last_change":"2023-06-23T18:03:37.768665+0000","last_active":"2023-06-23T18:03:42.916767+0000","last_peered":"2023-06-23T18:03:42.916767+0000","last_clean":"2023-06-23T18:03:42.916767+0000","last_became_active":"2023-06-23T18:03:37.768373+0000","last_became_peered":"2023-06-23T18:03:37.768373+0000","last_unstale":"2023-06-23T18:03:42.916767+0000","last_undegraded":"2023-06-23T18:03:42.916767+0000","last_fullsized":"2023-06-23T18:03:42.916767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:03:33.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923756+0000","last_change":"2023-06-23T18:03:37.376353+0000","last_active":"2023-06-23T18:03:42.923756+0000","last_peered":"2023-06-23T18:03:42.923756+0000","last_clean":"2023-06-23T18:03:42.923756+0000","last_became_active":"2023-06-23T18:03:37.375931+0000","last_became_peered":"2023-06-23T18:03:37.375931+0000","last_unstale":"2023-06-23T18:03:42.923756+0000","last_undegraded":"2023-06-23T18:03:42.923756+0000","last_fullsized":"2023-06-23T18:03:42.923756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:22.517180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920338+0000","last_change":"2023-06-23T18:03:41.787494+0000","last_active":"2023-06-23T18:03:42.920338+0000","last_peered":"2023-06-23T18:03:42.920338+0000","last_clean":"2023-06-23T18:03:42.920338+0000","last_became_active":"2023-06-23T18:03:41.787099+0000","last_became_peered":"2023-06-23T18:03:41.787099+0000","last_unstale":"2023-06-23T18:03:42.920338+0000","last_undegraded":"2023-06-23T18:03:42.920338+0000","last_fullsized":"2023-06-23T18:03:42.920338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922054+0000","last_change":"2023-06-23T18:03:37.377640+0000","last_active":"2023-06-23T18:03:42.922054+0000","last_peered":"2023-06-23T18:03:42.922054+0000","last_clean":"2023-06-23T18:03:42.922054+0000","last_became_active":"2023-06-23T18:03:37.377010+0000","last_became_peered":"2023-06-23T18:03:37.377010+0000","last_unstale":"2023-06-23T18:03:42.922054+0000","last_undegraded":"2023-06-23T18:03:42.922054+0000","last_fullsized":"2023-06-23T18:03:42.922054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917168+0000","last_change":"2023-06-23T18:03:37.756104+0000","last_active":"2023-06-23T18:03:42.917168+0000","last_peered":"2023-06-23T18:03:42.917168+0000","last_clean":"2023-06-23T18:03:42.917168+0000","last_became_active":"2023-06-23T18:03:37.755767+0000","last_became_peered":"2023-06-23T18:03:37.755767+0000","last_unstale":"2023-06-23T18:03:42.917168+0000","last_undegraded":"2023-06-23T18:03:42.917168+0000","last_fullsized":"2023-06-23T18:03:42.917168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:22:59.783746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917308+0000","last_change":"2023-06-23T18:03:41.785136+0000","last_active":"2023-06-23T18:03:42.917308+0000","last_peered":"2023-06-23T18:03:42.917308+0000","last_clean":"2023-06-23T18:03:42.917308+0000","last_became_active":"2023-06-23T18:03:41.784711+0000","last_became_peered":"2023-06-23T18:03:41.784711+0000","last_unstale":"2023-06-23T18:03:42.917308+0000","last_undegraded":"2023-06-23T18:03:42.917308+0000","last_fullsized":"2023-06-23T18:03:42.917308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:47.124821+0000","last_change":"2023-06-23T18:03:37.376017+0000","last_active":"2023-06-23T18:03:47.124821+0000","last_peered":"2023-06-23T18:03:47.124821+0000","last_clean":"2023-06-23T18:03:47.124821+0000","last_became_active":"2023-06-23T18:03:37.375728+0000","last_became_peered":"2023-06-23T18:03:37.375728+0000","last_unstale":"2023-06-23T18:03:47.124821+0000","last_undegraded":"2023-06-23T18:03:47.124821+0000","last_fullsized":"2023-06-23T18:03:47.124821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:57.309841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158169+0000","last_change":"2023-06-23T18:03:37.752163+0000","last_active":"2023-06-23T18:03:42.158169+0000","last_peered":"2023-06-23T18:03:42.158169+0000","last_clean":"2023-06-23T18:03:42.158169+0000","last_became_active":"2023-06-23T18:03:37.751393+0000","last_became_peered":"2023-06-23T18:03:37.751393+0000","last_unstale":"2023-06-23T18:03:42.158169+0000","last_undegraded":"2023-06-23T18:03:42.158169+0000","last_fullsized":"2023-06-23T18:03:42.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:54:13.905688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916838+0000","last_change":"2023-06-23T18:03:41.790305+0000","last_active":"2023-06-23T18:03:42.916838+0000","last_peered":"2023-06-23T18:03:42.916838+0000","last_clean":"2023-06-23T18:03:42.916838+0000","last_became_active":"2023-06-23T18:03:41.787857+0000","last_became_peered":"2023-06-23T18:03:41.787857+0000","last_unstale":"2023-06-23T18:03:42.916838+0000","last_undegraded":"2023-06-23T18:03:42.916838+0000","last_fullsized":"2023-06-23T18:03:42.916838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:54.379340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157126+0000","last_change":"2023-06-23T18:03:37.751336+0000","last_active":"2023-06-23T18:03:42.157126+0000","last_peered":"2023-06-23T18:03:42.157126+0000","last_clean":"2023-06-23T18:03:42.157126+0000","last_became_active":"2023-06-23T18:03:37.750981+0000","last_became_peered":"2023-06-23T18:03:37.750981+0000","last_unstale":"2023-06-23T18:03:42.157126+0000","last_undegraded":"2023-06-23T18:03:42.157126+0000","last_fullsized":"2023-06-23T18:03:42.157126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:17:15.424492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920331+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:03:42.920331+0000","last_peered":"2023-06-23T18:03:42.920331+0000","last_clean":"2023-06-23T18:03:42.920331+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:03:42.920331+0000","last_undegraded":"2023-06-23T18:03:42.920331+0000","last_fullsized":"2023-06-23T18:03:42.920331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:49:09.922836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922412+0000","last_change":"2023-06-23T18:03:37.375755+0000","last_active":"2023-06-23T18:03:42.922412+0000","last_peered":"2023-06-23T18:03:42.922412+0000","last_clean":"2023-06-23T18:03:42.922412+0000","last_became_active":"2023-06-23T18:03:37.375421+0000","last_became_peered":"2023-06-23T18:03:37.375421+0000","last_unstale":"2023-06-23T18:03:42.922412+0000","last_undegraded":"2023-06-23T18:03:42.922412+0000","last_fullsized":"2023-06-23T18:03:42.922412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915864+0000","last_change":"2023-06-23T18:03:41.785345+0000","last_active":"2023-06-23T18:03:42.915864+0000","last_peered":"2023-06-23T18:03:42.915864+0000","last_clean":"2023-06-23T18:03:42.915864+0000","last_became_active":"2023-06-23T18:03:41.784105+0000","last_became_peered":"2023-06-23T18:03:41.784105+0000","last_unstale":"2023-06-23T18:03:42.915864+0000","last_undegraded":"2023-06-23T18:03:42.915864+0000","last_fullsized":"2023-06-23T18:03:42.915864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:07:51.445920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915986+0000","last_change":"2023-06-23T18:03:37.768794+0000","last_active":"2023-06-23T18:03:42.915986+0000","last_peered":"2023-06-23T18:03:42.915986+0000","last_clean":"2023-06-23T18:03:42.915986+0000","last_became_active":"2023-06-23T18:03:37.768517+0000","last_became_peered":"2023-06-23T18:03:37.768517+0000","last_unstale":"2023-06-23T18:03:42.915986+0000","last_undegraded":"2023-06-23T18:03:42.915986+0000","last_fullsized":"2023-06-23T18:03:42.915986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:53.245906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920435+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:03:42.920435+0000","last_peered":"2023-06-23T18:03:42.920435+0000","last_clean":"2023-06-23T18:03:42.920435+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:03:42.920435+0000","last_undegraded":"2023-06-23T18:03:42.920435+0000","last_fullsized":"2023-06-23T18:03:42.920435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921748+0000","last_change":"2023-06-23T18:03:37.376156+0000","last_active":"2023-06-23T18:03:42.921748+0000","last_peered":"2023-06-23T18:03:42.921748+0000","last_clean":"2023-06-23T18:03:42.921748+0000","last_became_active":"2023-06-23T18:03:37.375870+0000","last_became_peered":"2023-06-23T18:03:37.375870+0000","last_unstale":"2023-06-23T18:03:42.921748+0000","last_undegraded":"2023-06-23T18:03:42.921748+0000","last_fullsized":"2023-06-23T18:03:42.921748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156557+0000","last_change":"2023-06-23T18:03:41.783782+0000","last_active":"2023-06-23T18:03:42.156557+0000","last_peered":"2023-06-23T18:03:42.156557+0000","last_clean":"2023-06-23T18:03:42.156557+0000","last_became_active":"2023-06-23T18:03:41.783336+0000","last_became_peered":"2023-06-23T18:03:41.783336+0000","last_unstale":"2023-06-23T18:03:42.156557+0000","last_undegraded":"2023-06-23T18:03:42.156557+0000","last_fullsized":"2023-06-23T18:03:42.156557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:55.511609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920320+0000","last_change":"2023-06-23T18:03:37.755707+0000","last_active":"2023-06-23T18:03:42.920320+0000","last_peered":"2023-06-23T18:03:42.920320+0000","last_clean":"2023-06-23T18:03:42.920320+0000","last_became_active":"2023-06-23T18:03:37.755413+0000","last_became_peered":"2023-06-23T18:03:37.755413+0000","last_unstale":"2023-06-23T18:03:42.920320+0000","last_undegraded":"2023-06-23T18:03:42.920320+0000","last_fullsized":"2023-06-23T18:03:42.920320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922407+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:03:42.922407+0000","last_peered":"2023-06-23T18:03:42.922407+0000","last_clean":"2023-06-23T18:03:42.922407+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:03:42.922407+0000","last_undegraded":"2023-06-23T18:03:42.922407+0000","last_fullsized":"2023-06-23T18:03:42.922407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:55.340944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918354+0000","last_change":"2023-06-23T18:03:37.370271+0000","last_active":"2023-06-23T18:03:42.918354+0000","last_peered":"2023-06-23T18:03:42.918354+0000","last_clean":"2023-06-23T18:03:42.918354+0000","last_became_active":"2023-06-23T18:03:37.369994+0000","last_became_peered":"2023-06-23T18:03:37.369994+0000","last_unstale":"2023-06-23T18:03:42.918354+0000","last_undegraded":"2023-06-23T18:03:42.918354+0000","last_fullsized":"2023-06-23T18:03:42.918354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:36.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920193+0000","last_change":"2023-06-23T18:03:41.793568+0000","last_active":"2023-06-23T18:03:42.920193+0000","last_peered":"2023-06-23T18:03:42.920193+0000","last_clean":"2023-06-23T18:03:42.920193+0000","last_became_active":"2023-06-23T18:03:41.793190+0000","last_became_peered":"2023-06-23T18:03:41.793190+0000","last_unstale":"2023-06-23T18:03:42.920193+0000","last_undegraded":"2023-06-23T18:03:42.920193+0000","last_fullsized":"2023-06-23T18:03:42.920193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:35.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917232+0000","last_change":"2023-06-23T18:03:37.764627+0000","last_active":"2023-06-23T18:03:42.917232+0000","last_peered":"2023-06-23T18:03:42.917232+0000","last_clean":"2023-06-23T18:03:42.917232+0000","last_became_active":"2023-06-23T18:03:37.764255+0000","last_became_peered":"2023-06-23T18:03:37.764255+0000","last_unstale":"2023-06-23T18:03:42.917232+0000","last_undegraded":"2023-06-23T18:03:42.917232+0000","last_fullsized":"2023-06-23T18:03:42.917232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918569+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:03:42.918569+0000","last_peered":"2023-06-23T18:03:42.918569+0000","last_clean":"2023-06-23T18:03:42.918569+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:03:42.918569+0000","last_undegraded":"2023-06-23T18:03:42.918569+0000","last_fullsized":"2023-06-23T18:03:42.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:25.233307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924293+0000","last_change":"2023-06-23T18:03:37.371127+0000","last_active":"2023-06-23T18:03:42.924293+0000","last_peered":"2023-06-23T18:03:42.924293+0000","last_clean":"2023-06-23T18:03:42.924293+0000","last_became_active":"2023-06-23T18:03:37.370810+0000","last_became_peered":"2023-06-23T18:03:37.370810+0000","last_unstale":"2023-06-23T18:03:42.924293+0000","last_undegraded":"2023-06-23T18:03:42.924293+0000","last_fullsized":"2023-06-23T18:03:42.924293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:42.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919827+0000","last_change":"2023-06-23T18:03:41.791645+0000","last_active":"2023-06-23T18:03:42.919827+0000","last_peered":"2023-06-23T18:03:42.919827+0000","last_clean":"2023-06-23T18:03:42.919827+0000","last_became_active":"2023-06-23T18:03:41.791276+0000","last_became_peered":"2023-06-23T18:03:41.791276+0000","last_unstale":"2023-06-23T18:03:42.919827+0000","last_undegraded":"2023-06-23T18:03:42.919827+0000","last_fullsized":"2023-06-23T18:03:42.919827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:24.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918088+0000","last_change":"2023-06-23T18:03:37.754565+0000","last_active":"2023-06-23T18:03:42.918088+0000","last_peered":"2023-06-23T18:03:42.918088+0000","last_clean":"2023-06-23T18:03:42.918088+0000","last_became_active":"2023-06-23T18:03:37.754281+0000","last_became_peered":"2023-06-23T18:03:37.754281+0000","last_unstale":"2023-06-23T18:03:42.918088+0000","last_undegraded":"2023-06-23T18:03:42.918088+0000","last_fullsized":"2023-06-23T18:03:42.918088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158267+0000","last_change":"2023-06-23T18:03:37.381149+0000","last_active":"2023-06-23T18:03:42.158267+0000","last_peered":"2023-06-23T18:03:42.158267+0000","last_clean":"2023-06-23T18:03:42.158267+0000","last_became_active":"2023-06-23T18:03:37.380831+0000","last_became_peered":"2023-06-23T18:03:37.380831+0000","last_unstale":"2023-06-23T18:03:42.158267+0000","last_undegraded":"2023-06-23T18:03:42.158267+0000","last_fullsized":"2023-06-23T18:03:42.158267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:40:27.306991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923558+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:03:42.923558+0000","last_peered":"2023-06-23T18:03:42.923558+0000","last_clean":"2023-06-23T18:03:42.923558+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:03:42.923558+0000","last_undegraded":"2023-06-23T18:03:42.923558+0000","last_fullsized":"2023-06-23T18:03:42.923558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921567+0000","last_change":"2023-06-23T18:03:41.787633+0000","last_active":"2023-06-23T18:03:42.921567+0000","last_peered":"2023-06-23T18:03:42.921567+0000","last_clean":"2023-06-23T18:03:42.921567+0000","last_became_active":"2023-06-23T18:03:41.786906+0000","last_became_peered":"2023-06-23T18:03:41.786906+0000","last_unstale":"2023-06-23T18:03:42.921567+0000","last_undegraded":"2023-06-23T18:03:42.921567+0000","last_fullsized":"2023-06-23T18:03:42.921567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919645+0000","last_change":"2023-06-23T18:03:37.764949+0000","last_active":"2023-06-23T18:03:42.919645+0000","last_peered":"2023-06-23T18:03:42.919645+0000","last_clean":"2023-06-23T18:03:42.919645+0000","last_became_active":"2023-06-23T18:03:37.764358+0000","last_became_peered":"2023-06-23T18:03:37.764358+0000","last_unstale":"2023-06-23T18:03:42.919645+0000","last_undegraded":"2023-06-23T18:03:42.919645+0000","last_fullsized":"2023-06-23T18:03:42.919645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916086+0000","last_change":"2023-06-23T18:03:37.371283+0000","last_active":"2023-06-23T18:03:42.916086+0000","last_peered":"2023-06-23T18:03:42.916086+0000","last_clean":"2023-06-23T18:03:42.916086+0000","last_became_active":"2023-06-23T18:03:37.370358+0000","last_became_peered":"2023-06-23T18:03:37.370358+0000","last_unstale":"2023-06-23T18:03:42.916086+0000","last_undegraded":"2023-06-23T18:03:42.916086+0000","last_fullsized":"2023-06-23T18:03:42.916086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152204+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:03:42.152204+0000","last_peered":"2023-06-23T18:03:42.152204+0000","last_clean":"2023-06-23T18:03:42.152204+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:03:42.152204+0000","last_undegraded":"2023-06-23T18:03:42.152204+0000","last_fullsized":"2023-06-23T18:03:42.152204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:44.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152495+0000","last_change":"2023-06-23T18:03:41.787323+0000","last_active":"2023-06-23T18:03:42.152495+0000","last_peered":"2023-06-23T18:03:42.152495+0000","last_clean":"2023-06-23T18:03:42.152495+0000","last_became_active":"2023-06-23T18:03:41.786742+0000","last_became_peered":"2023-06-23T18:03:41.786742+0000","last_unstale":"2023-06-23T18:03:42.152495+0000","last_undegraded":"2023-06-23T18:03:42.152495+0000","last_fullsized":"2023-06-23T18:03:42.152495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:39:47.931343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922227+0000","last_change":"2023-06-23T18:03:37.743506+0000","last_active":"2023-06-23T18:03:42.922227+0000","last_peered":"2023-06-23T18:03:42.922227+0000","last_clean":"2023-06-23T18:03:42.922227+0000","last_became_active":"2023-06-23T18:03:37.743222+0000","last_became_peered":"2023-06-23T18:03:37.743222+0000","last_unstale":"2023-06-23T18:03:42.922227+0000","last_undegraded":"2023-06-23T18:03:42.922227+0000","last_fullsized":"2023-06-23T18:03:42.922227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:43.133522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920271+0000","last_change":"2023-06-23T18:03:37.375782+0000","last_active":"2023-06-23T18:03:42.920271+0000","last_peered":"2023-06-23T18:03:42.920271+0000","last_clean":"2023-06-23T18:03:42.920271+0000","last_became_active":"2023-06-23T18:03:37.375451+0000","last_became_peered":"2023-06-23T18:03:37.375451+0000","last_unstale":"2023-06-23T18:03:42.920271+0000","last_undegraded":"2023-06-23T18:03:42.920271+0000","last_fullsized":"2023-06-23T18:03:42.920271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:35.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921282+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:03:42.921282+0000","last_peered":"2023-06-23T18:03:42.921282+0000","last_clean":"2023-06-23T18:03:42.921282+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:03:42.921282+0000","last_undegraded":"2023-06-23T18:03:42.921282+0000","last_fullsized":"2023-06-23T18:03:42.921282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922998+0000","last_change":"2023-06-23T18:03:41.788944+0000","last_active":"2023-06-23T18:03:42.922998+0000","last_peered":"2023-06-23T18:03:42.922998+0000","last_clean":"2023-06-23T18:03:42.922998+0000","last_became_active":"2023-06-23T18:03:41.785916+0000","last_became_peered":"2023-06-23T18:03:41.785916+0000","last_unstale":"2023-06-23T18:03:42.922998+0000","last_undegraded":"2023-06-23T18:03:42.922998+0000","last_fullsized":"2023-06-23T18:03:42.922998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919919+0000","last_change":"2023-06-23T18:03:37.750749+0000","last_active":"2023-06-23T18:03:42.919919+0000","last_peered":"2023-06-23T18:03:42.919919+0000","last_clean":"2023-06-23T18:03:42.919919+0000","last_became_active":"2023-06-23T18:03:37.750424+0000","last_became_peered":"2023-06-23T18:03:37.750424+0000","last_unstale":"2023-06-23T18:03:42.919919+0000","last_undegraded":"2023-06-23T18:03:42.919919+0000","last_fullsized":"2023-06-23T18:03:42.919919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:10.920865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917678+0000","last_change":"2023-06-23T18:03:37.377053+0000","last_active":"2023-06-23T18:03:42.917678+0000","last_peered":"2023-06-23T18:03:42.917678+0000","last_clean":"2023-06-23T18:03:42.917678+0000","last_became_active":"2023-06-23T18:03:37.376770+0000","last_became_peered":"2023-06-23T18:03:37.376770+0000","last_unstale":"2023-06-23T18:03:42.917678+0000","last_undegraded":"2023-06-23T18:03:42.917678+0000","last_fullsized":"2023-06-23T18:03:42.917678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923092+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:03:42.923092+0000","last_peered":"2023-06-23T18:03:42.923092+0000","last_clean":"2023-06-23T18:03:42.923092+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:03:42.923092+0000","last_undegraded":"2023-06-23T18:03:42.923092+0000","last_fullsized":"2023-06-23T18:03:42.923092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:00.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921588+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:03:42.921588+0000","last_peered":"2023-06-23T18:03:42.921588+0000","last_clean":"2023-06-23T18:03:42.921588+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:03:42.921588+0000","last_undegraded":"2023-06-23T18:03:42.921588+0000","last_fullsized":"2023-06-23T18:03:42.921588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921628+0000","last_change":"2023-06-23T18:03:41.792372+0000","last_active":"2023-06-23T18:03:42.921628+0000","last_peered":"2023-06-23T18:03:42.921628+0000","last_clean":"2023-06-23T18:03:42.921628+0000","last_became_active":"2023-06-23T18:03:41.791937+0000","last_became_peered":"2023-06-23T18:03:41.791937+0000","last_unstale":"2023-06-23T18:03:42.921628+0000","last_undegraded":"2023-06-23T18:03:42.921628+0000","last_fullsized":"2023-06-23T18:03:42.921628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:26.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919683+0000","last_change":"2023-06-23T18:03:37.764721+0000","last_active":"2023-06-23T18:03:42.919683+0000","last_peered":"2023-06-23T18:03:42.919683+0000","last_clean":"2023-06-23T18:03:42.919683+0000","last_became_active":"2023-06-23T18:03:37.755578+0000","last_became_peered":"2023-06-23T18:03:37.755578+0000","last_unstale":"2023-06-23T18:03:42.919683+0000","last_undegraded":"2023-06-23T18:03:42.919683+0000","last_fullsized":"2023-06-23T18:03:42.919683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916131+0000","last_change":"2023-06-23T18:03:37.369824+0000","last_active":"2023-06-23T18:03:42.916131+0000","last_peered":"2023-06-23T18:03:42.916131+0000","last_clean":"2023-06-23T18:03:42.916131+0000","last_became_active":"2023-06-23T18:03:37.369094+0000","last_became_peered":"2023-06-23T18:03:37.369094+0000","last_unstale":"2023-06-23T18:03:42.916131+0000","last_undegraded":"2023-06-23T18:03:42.916131+0000","last_fullsized":"2023-06-23T18:03:42.916131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:53.895398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159093+0000","last_change":"2023-06-23T18:03:41.789799+0000","last_active":"2023-06-23T18:03:42.159093+0000","last_peered":"2023-06-23T18:03:42.159093+0000","last_clean":"2023-06-23T18:03:42.159093+0000","last_became_active":"2023-06-23T18:03:41.789426+0000","last_became_peered":"2023-06-23T18:03:41.789426+0000","last_unstale":"2023-06-23T18:03:42.159093+0000","last_undegraded":"2023-06-23T18:03:42.159093+0000","last_fullsized":"2023-06-23T18:03:42.159093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:36:43.668342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159103+0000","last_change":"2023-06-23T18:03:37.751763+0000","last_active":"2023-06-23T18:03:42.159103+0000","last_peered":"2023-06-23T18:03:42.159103+0000","last_clean":"2023-06-23T18:03:42.159103+0000","last_became_active":"2023-06-23T18:03:37.751310+0000","last_became_peered":"2023-06-23T18:03:37.751310+0000","last_unstale":"2023-06-23T18:03:42.159103+0000","last_undegraded":"2023-06-23T18:03:42.159103+0000","last_fullsized":"2023-06-23T18:03:42.159103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:00.581207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157189+0000","last_change":"2023-06-23T18:03:37.373425+0000","last_active":"2023-06-23T18:03:42.157189+0000","last_peered":"2023-06-23T18:03:42.157189+0000","last_clean":"2023-06-23T18:03:42.157189+0000","last_became_active":"2023-06-23T18:03:37.373154+0000","last_became_peered":"2023-06-23T18:03:37.373154+0000","last_unstale":"2023-06-23T18:03:42.157189+0000","last_undegraded":"2023-06-23T18:03:42.157189+0000","last_fullsized":"2023-06-23T18:03:42.157189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:33.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922639+0000","last_change":"2023-06-23T18:03:41.792127+0000","last_active":"2023-06-23T18:03:42.922639+0000","last_peered":"2023-06-23T18:03:42.922639+0000","last_clean":"2023-06-23T18:03:42.922639+0000","last_became_active":"2023-06-23T18:03:41.789481+0000","last_became_peered":"2023-06-23T18:03:41.789481+0000","last_unstale":"2023-06-23T18:03:42.922639+0000","last_undegraded":"2023-06-23T18:03:42.922639+0000","last_fullsized":"2023-06-23T18:03:42.922639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158754+0000","last_change":"2023-06-23T18:03:37.754662+0000","last_active":"2023-06-23T18:03:42.158754+0000","last_peered":"2023-06-23T18:03:42.158754+0000","last_clean":"2023-06-23T18:03:42.158754+0000","last_became_active":"2023-06-23T18:03:37.753982+0000","last_became_peered":"2023-06-23T18:03:37.753982+0000","last_unstale":"2023-06-23T18:03:42.158754+0000","last_undegraded":"2023-06-23T18:03:42.158754+0000","last_fullsized":"2023-06-23T18:03:42.158754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:40.534179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921128+0000","last_change":"2023-06-23T18:03:37.380222+0000","last_active":"2023-06-23T18:03:42.921128+0000","last_peered":"2023-06-23T18:03:42.921128+0000","last_clean":"2023-06-23T18:03:42.921128+0000","last_became_active":"2023-06-23T18:03:37.379866+0000","last_became_peered":"2023-06-23T18:03:37.379866+0000","last_unstale":"2023-06-23T18:03:42.921128+0000","last_undegraded":"2023-06-23T18:03:42.921128+0000","last_fullsized":"2023-06-23T18:03:42.921128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:12.333579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917855+0000","last_change":"2023-06-23T18:03:41.782310+0000","last_active":"2023-06-23T18:03:42.917855+0000","last_peered":"2023-06-23T18:03:42.917855+0000","last_clean":"2023-06-23T18:03:42.917855+0000","last_became_active":"2023-06-23T18:03:41.781820+0000","last_became_peered":"2023-06-23T18:03:41.781820+0000","last_unstale":"2023-06-23T18:03:42.917855+0000","last_undegraded":"2023-06-23T18:03:42.917855+0000","last_fullsized":"2023-06-23T18:03:42.917855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:39.845448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158787+0000","last_change":"2023-06-23T18:03:37.753742+0000","last_active":"2023-06-23T18:03:42.158787+0000","last_peered":"2023-06-23T18:03:42.158787+0000","last_clean":"2023-06-23T18:03:42.158787+0000","last_became_active":"2023-06-23T18:03:37.753391+0000","last_became_peered":"2023-06-23T18:03:37.753391+0000","last_unstale":"2023-06-23T18:03:42.158787+0000","last_undegraded":"2023-06-23T18:03:42.158787+0000","last_fullsized":"2023-06-23T18:03:42.158787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:54.787520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918436+0000","last_change":"2023-06-23T18:03:37.378501+0000","last_active":"2023-06-23T18:03:42.918436+0000","last_peered":"2023-06-23T18:03:42.918436+0000","last_clean":"2023-06-23T18:03:42.918436+0000","last_became_active":"2023-06-23T18:03:37.378222+0000","last_became_peered":"2023-06-23T18:03:37.378222+0000","last_unstale":"2023-06-23T18:03:42.918436+0000","last_undegraded":"2023-06-23T18:03:42.918436+0000","last_fullsized":"2023-06-23T18:03:42.918436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:49.965045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152626+0000","last_change":"2023-06-23T18:03:41.788999+0000","last_active":"2023-06-23T18:03:42.152626+0000","last_peered":"2023-06-23T18:03:42.152626+0000","last_clean":"2023-06-23T18:03:42.152626+0000","last_became_active":"2023-06-23T18:03:41.788606+0000","last_became_peered":"2023-06-23T18:03:41.788606+0000","last_unstale":"2023-06-23T18:03:42.152626+0000","last_undegraded":"2023-06-23T18:03:42.152626+0000","last_fullsized":"2023-06-23T18:03:42.152626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:40:16.265229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920805+0000","last_change":"2023-06-23T18:03:37.751267+0000","last_active":"2023-06-23T18:03:42.920805+0000","last_peered":"2023-06-23T18:03:42.920805+0000","last_clean":"2023-06-23T18:03:42.920805+0000","last_became_active":"2023-06-23T18:03:37.746037+0000","last_became_peered":"2023-06-23T18:03:37.746037+0000","last_unstale":"2023-06-23T18:03:42.920805+0000","last_undegraded":"2023-06-23T18:03:42.920805+0000","last_fullsized":"2023-06-23T18:03:42.920805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922834+0000","last_change":"2023-06-23T18:03:37.369505+0000","last_active":"2023-06-23T18:03:42.922834+0000","last_peered":"2023-06-23T18:03:42.922834+0000","last_clean":"2023-06-23T18:03:42.922834+0000","last_became_active":"2023-06-23T18:03:37.369174+0000","last_became_peered":"2023-06-23T18:03:37.369174+0000","last_unstale":"2023-06-23T18:03:42.922834+0000","last_undegraded":"2023-06-23T18:03:42.922834+0000","last_fullsized":"2023-06-23T18:03:42.922834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151984+0000","last_change":"2023-06-23T18:03:41.781321+0000","last_active":"2023-06-23T18:03:42.151984+0000","last_peered":"2023-06-23T18:03:42.151984+0000","last_clean":"2023-06-23T18:03:42.151984+0000","last_became_active":"2023-06-23T18:03:41.780831+0000","last_became_peered":"2023-06-23T18:03:41.780831+0000","last_unstale":"2023-06-23T18:03:42.151984+0000","last_undegraded":"2023-06-23T18:03:42.151984+0000","last_fullsized":"2023-06-23T18:03:42.151984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:53.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920510+0000","last_change":"2023-06-23T18:03:37.761108+0000","last_active":"2023-06-23T18:03:42.920510+0000","last_peered":"2023-06-23T18:03:42.920510+0000","last_clean":"2023-06-23T18:03:42.920510+0000","last_became_active":"2023-06-23T18:03:37.760041+0000","last_became_peered":"2023-06-23T18:03:37.760041+0000","last_unstale":"2023-06-23T18:03:42.920510+0000","last_undegraded":"2023-06-23T18:03:42.920510+0000","last_fullsized":"2023-06-23T18:03:42.920510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:46:38.396933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922050+0000","last_change":"2023-06-23T18:03:37.377838+0000","last_active":"2023-06-23T18:03:42.922050+0000","last_peered":"2023-06-23T18:03:42.922050+0000","last_clean":"2023-06-23T18:03:42.922050+0000","last_became_active":"2023-06-23T18:03:37.377381+0000","last_became_peered":"2023-06-23T18:03:37.377381+0000","last_unstale":"2023-06-23T18:03:42.922050+0000","last_undegraded":"2023-06-23T18:03:42.922050+0000","last_fullsized":"2023-06-23T18:03:42.922050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152297+0000","last_change":"2023-06-23T18:03:41.787926+0000","last_active":"2023-06-23T18:03:42.152297+0000","last_peered":"2023-06-23T18:03:42.152297+0000","last_clean":"2023-06-23T18:03:42.152297+0000","last_became_active":"2023-06-23T18:03:41.787539+0000","last_became_peered":"2023-06-23T18:03:41.787539+0000","last_unstale":"2023-06-23T18:03:42.152297+0000","last_undegraded":"2023-06-23T18:03:42.152297+0000","last_fullsized":"2023-06-23T18:03:42.152297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:49.966258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156824+0000","last_change":"2023-06-23T18:03:37.745296+0000","last_active":"2023-06-23T18:03:42.156824+0000","last_peered":"2023-06-23T18:03:42.156824+0000","last_clean":"2023-06-23T18:03:42.156824+0000","last_became_active":"2023-06-23T18:03:37.744996+0000","last_became_peered":"2023-06-23T18:03:37.744996+0000","last_unstale":"2023-06-23T18:03:42.156824+0000","last_undegraded":"2023-06-23T18:03:42.156824+0000","last_fullsized":"2023-06-23T18:03:42.156824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:07:37.054289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922271+0000","last_change":"2023-06-23T18:03:37.381405+0000","last_active":"2023-06-23T18:03:42.922271+0000","last_peered":"2023-06-23T18:03:42.922271+0000","last_clean":"2023-06-23T18:03:42.922271+0000","last_became_active":"2023-06-23T18:03:37.381038+0000","last_became_peered":"2023-06-23T18:03:37.381038+0000","last_unstale":"2023-06-23T18:03:42.922271+0000","last_undegraded":"2023-06-23T18:03:42.922271+0000","last_fullsized":"2023-06-23T18:03:42.922271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151761+0000","last_change":"2023-06-23T18:03:41.787211+0000","last_active":"2023-06-23T18:03:42.151761+0000","last_peered":"2023-06-23T18:03:42.151761+0000","last_clean":"2023-06-23T18:03:42.151761+0000","last_became_active":"2023-06-23T18:03:41.786809+0000","last_became_peered":"2023-06-23T18:03:41.786809+0000","last_unstale":"2023-06-23T18:03:42.151761+0000","last_undegraded":"2023-06-23T18:03:42.151761+0000","last_fullsized":"2023-06-23T18:03:42.151761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:42.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919723+0000","last_change":"2023-06-23T18:03:37.770649+0000","last_active":"2023-06-23T18:03:42.919723+0000","last_peered":"2023-06-23T18:03:42.919723+0000","last_clean":"2023-06-23T18:03:42.919723+0000","last_became_active":"2023-06-23T18:03:37.770326+0000","last_became_peered":"2023-06-23T18:03:37.770326+0000","last_unstale":"2023-06-23T18:03:42.919723+0000","last_undegraded":"2023-06-23T18:03:42.919723+0000","last_fullsized":"2023-06-23T18:03:42.919723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923872+0000","last_change":"2023-06-23T18:03:37.381617+0000","last_active":"2023-06-23T18:03:42.923872+0000","last_peered":"2023-06-23T18:03:42.923872+0000","last_clean":"2023-06-23T18:03:42.923872+0000","last_became_active":"2023-06-23T18:03:37.381342+0000","last_became_peered":"2023-06-23T18:03:37.381342+0000","last_unstale":"2023-06-23T18:03:42.923872+0000","last_undegraded":"2023-06-23T18:03:42.923872+0000","last_fullsized":"2023-06-23T18:03:42.923872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:31.692075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920724+0000","last_change":"2023-06-23T18:03:41.791679+0000","last_active":"2023-06-23T18:03:42.920724+0000","last_peered":"2023-06-23T18:03:42.920724+0000","last_clean":"2023-06-23T18:03:42.920724+0000","last_became_active":"2023-06-23T18:03:41.788101+0000","last_became_peered":"2023-06-23T18:03:41.788101+0000","last_unstale":"2023-06-23T18:03:42.920724+0000","last_undegraded":"2023-06-23T18:03:42.920724+0000","last_fullsized":"2023-06-23T18:03:42.920724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924128+0000","last_change":"2023-06-23T18:03:41.789672+0000","last_active":"2023-06-23T18:03:42.924128+0000","last_peered":"2023-06-23T18:03:42.924128+0000","last_clean":"2023-06-23T18:03:42.924128+0000","last_became_active":"2023-06-23T18:03:41.789258+0000","last_became_peered":"2023-06-23T18:03:41.789258+0000","last_unstale":"2023-06-23T18:03:42.924128+0000","last_undegraded":"2023-06-23T18:03:42.924128+0000","last_fullsized":"2023-06-23T18:03:42.924128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:35:03.864692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919206+0000","last_change":"2023-06-23T18:03:37.380523+0000","last_active":"2023-06-23T18:03:42.919206+0000","last_peered":"2023-06-23T18:03:42.919206+0000","last_clean":"2023-06-23T18:03:42.919206+0000","last_became_active":"2023-06-23T18:03:37.380210+0000","last_became_peered":"2023-06-23T18:03:37.380210+0000","last_unstale":"2023-06-23T18:03:42.919206+0000","last_undegraded":"2023-06-23T18:03:42.919206+0000","last_fullsized":"2023-06-23T18:03:42.919206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:57.050399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921568+0000","last_change":"2023-06-23T18:03:37.769174+0000","last_active":"2023-06-23T18:03:42.921568+0000","last_peered":"2023-06-23T18:03:42.921568+0000","last_clean":"2023-06-23T18:03:42.921568+0000","last_became_active":"2023-06-23T18:03:37.768922+0000","last_became_peered":"2023-06-23T18:03:37.768922+0000","last_unstale":"2023-06-23T18:03:42.921568+0000","last_undegraded":"2023-06-23T18:03:42.921568+0000","last_fullsized":"2023-06-23T18:03:42.921568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921045+0000","last_change":"2023-06-23T18:03:41.785038+0000","last_active":"2023-06-23T18:03:42.921045+0000","last_peered":"2023-06-23T18:03:42.921045+0000","last_clean":"2023-06-23T18:03:42.921045+0000","last_became_active":"2023-06-23T18:03:41.784124+0000","last_became_peered":"2023-06-23T18:03:41.784124+0000","last_unstale":"2023-06-23T18:03:42.921045+0000","last_undegraded":"2023-06-23T18:03:42.921045+0000","last_fullsized":"2023-06-23T18:03:42.921045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921205+0000","last_change":"2023-06-23T18:03:37.381450+0000","last_active":"2023-06-23T18:03:42.921205+0000","last_peered":"2023-06-23T18:03:42.921205+0000","last_clean":"2023-06-23T18:03:42.921205+0000","last_became_active":"2023-06-23T18:03:37.381137+0000","last_became_peered":"2023-06-23T18:03:37.381137+0000","last_unstale":"2023-06-23T18:03:42.921205+0000","last_undegraded":"2023-06-23T18:03:42.921205+0000","last_fullsized":"2023-06-23T18:03:42.921205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:04.690816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922338+0000","last_change":"2023-06-23T18:03:37.752538+0000","last_active":"2023-06-23T18:03:42.922338+0000","last_peered":"2023-06-23T18:03:42.922338+0000","last_clean":"2023-06-23T18:03:42.922338+0000","last_became_active":"2023-06-23T18:03:37.752276+0000","last_became_peered":"2023-06-23T18:03:37.752276+0000","last_unstale":"2023-06-23T18:03:42.922338+0000","last_undegraded":"2023-06-23T18:03:42.922338+0000","last_fullsized":"2023-06-23T18:03:42.922338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:23.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921015+0000","last_change":"2023-06-23T18:03:37.376067+0000","last_active":"2023-06-23T18:03:42.921015+0000","last_peered":"2023-06-23T18:03:42.921015+0000","last_clean":"2023-06-23T18:03:42.921015+0000","last_became_active":"2023-06-23T18:03:37.375582+0000","last_became_peered":"2023-06-23T18:03:37.375582+0000","last_unstale":"2023-06-23T18:03:42.921015+0000","last_undegraded":"2023-06-23T18:03:42.921015+0000","last_fullsized":"2023-06-23T18:03:42.921015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924004+0000","last_change":"2023-06-23T18:03:37.770794+0000","last_active":"2023-06-23T18:03:42.924004+0000","last_peered":"2023-06-23T18:03:42.924004+0000","last_clean":"2023-06-23T18:03:42.924004+0000","last_became_active":"2023-06-23T18:03:37.770508+0000","last_became_peered":"2023-06-23T18:03:37.770508+0000","last_unstale":"2023-06-23T18:03:42.924004+0000","last_undegraded":"2023-06-23T18:03:42.924004+0000","last_fullsized":"2023-06-23T18:03:42.924004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:27.966872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919085+0000","last_change":"2023-06-23T18:03:41.784964+0000","last_active":"2023-06-23T18:03:42.919085+0000","last_peered":"2023-06-23T18:03:42.919085+0000","last_clean":"2023-06-23T18:03:42.919085+0000","last_became_active":"2023-06-23T18:03:41.783875+0000","last_became_peered":"2023-06-23T18:03:41.783875+0000","last_unstale":"2023-06-23T18:03:42.919085+0000","last_undegraded":"2023-06-23T18:03:42.919085+0000","last_fullsized":"2023-06-23T18:03:42.919085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:03.189212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921280+0000","last_change":"2023-06-23T18:03:37.381471+0000","last_active":"2023-06-23T18:03:42.921280+0000","last_peered":"2023-06-23T18:03:42.921280+0000","last_clean":"2023-06-23T18:03:42.921280+0000","last_became_active":"2023-06-23T18:03:37.381080+0000","last_became_peered":"2023-06-23T18:03:37.381080+0000","last_unstale":"2023-06-23T18:03:42.921280+0000","last_undegraded":"2023-06-23T18:03:42.921280+0000","last_fullsized":"2023-06-23T18:03:42.921280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:36.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919682+0000","last_change":"2023-06-23T18:03:37.770191+0000","last_active":"2023-06-23T18:03:42.919682+0000","last_peered":"2023-06-23T18:03:42.919682+0000","last_clean":"2023-06-23T18:03:42.919682+0000","last_became_active":"2023-06-23T18:03:37.769918+0000","last_became_peered":"2023-06-23T18:03:37.769918+0000","last_unstale":"2023-06-23T18:03:42.919682+0000","last_undegraded":"2023-06-23T18:03:42.919682+0000","last_fullsized":"2023-06-23T18:03:42.919682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:36.439982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924483+0000","last_change":"2023-06-23T18:03:41.789295+0000","last_active":"2023-06-23T18:03:42.924483+0000","last_peered":"2023-06-23T18:03:42.924483+0000","last_clean":"2023-06-23T18:03:42.924483+0000","last_became_active":"2023-06-23T18:03:41.788890+0000","last_became_peered":"2023-06-23T18:03:41.788890+0000","last_unstale":"2023-06-23T18:03:42.924483+0000","last_undegraded":"2023-06-23T18:03:42.924483+0000","last_fullsized":"2023-06-23T18:03:42.924483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:36.106369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918865+0000","last_change":"2023-06-23T18:03:37.759952+0000","last_active":"2023-06-23T18:03:42.918865+0000","last_peered":"2023-06-23T18:03:42.918865+0000","last_clean":"2023-06-23T18:03:42.918865+0000","last_became_active":"2023-06-23T18:03:37.759685+0000","last_became_peered":"2023-06-23T18:03:37.759685+0000","last_unstale":"2023-06-23T18:03:42.918865+0000","last_undegraded":"2023-06-23T18:03:42.918865+0000","last_fullsized":"2023-06-23T18:03:42.918865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:01:11.356694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916611+0000","last_change":"2023-06-23T18:03:37.761169+0000","last_active":"2023-06-23T18:03:42.916611+0000","last_peered":"2023-06-23T18:03:42.916611+0000","last_clean":"2023-06-23T18:03:42.916611+0000","last_became_active":"2023-06-23T18:03:37.760915+0000","last_became_peered":"2023-06-23T18:03:37.760915+0000","last_unstale":"2023-06-23T18:03:42.916611+0000","last_undegraded":"2023-06-23T18:03:42.916611+0000","last_fullsized":"2023-06-23T18:03:42.916611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:15:41.693241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157660+0000","last_change":"2023-06-23T18:03:37.756608+0000","last_active":"2023-06-23T18:03:42.157660+0000","last_peered":"2023-06-23T18:03:42.157660+0000","last_clean":"2023-06-23T18:03:42.157660+0000","last_became_active":"2023-06-23T18:03:37.756267+0000","last_became_peered":"2023-06-23T18:03:37.756267+0000","last_unstale":"2023-06-23T18:03:42.157660+0000","last_undegraded":"2023-06-23T18:03:42.157660+0000","last_fullsized":"2023-06-23T18:03:42.157660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:39:56.896505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917984+0000","last_change":"2023-06-23T18:03:37.768762+0000","last_active":"2023-06-23T18:03:42.917984+0000","last_peered":"2023-06-23T18:03:42.917984+0000","last_clean":"2023-06-23T18:03:42.917984+0000","last_became_active":"2023-06-23T18:03:37.768493+0000","last_became_peered":"2023-06-23T18:03:37.768493+0000","last_unstale":"2023-06-23T18:03:42.917984+0000","last_undegraded":"2023-06-23T18:03:42.917984+0000","last_fullsized":"2023-06-23T18:03:42.917984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:47.275516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918340+0000","last_change":"2023-06-23T18:03:37.770023+0000","last_active":"2023-06-23T18:03:42.918340+0000","last_peered":"2023-06-23T18:03:42.918340+0000","last_clean":"2023-06-23T18:03:42.918340+0000","last_became_active":"2023-06-23T18:03:37.769776+0000","last_became_peered":"2023-06-23T18:03:37.769776+0000","last_unstale":"2023-06-23T18:03:42.918340+0000","last_undegraded":"2023-06-23T18:03:42.918340+0000","last_fullsized":"2023-06-23T18:03:42.918340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923845+0000","last_change":"2023-06-23T18:03:37.772405+0000","last_active":"2023-06-23T18:03:42.923845+0000","last_peered":"2023-06-23T18:03:42.923845+0000","last_clean":"2023-06-23T18:03:42.923845+0000","last_became_active":"2023-06-23T18:03:37.772086+0000","last_became_peered":"2023-06-23T18:03:37.772086+0000","last_unstale":"2023-06-23T18:03:42.923845+0000","last_undegraded":"2023-06-23T18:03:42.923845+0000","last_fullsized":"2023-06-23T18:03:42.923845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:49.758628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919917+0000","last_change":"2023-06-23T18:03:37.768948+0000","last_active":"2023-06-23T18:03:42.919917+0000","last_peered":"2023-06-23T18:03:42.919917+0000","last_clean":"2023-06-23T18:03:42.919917+0000","last_became_active":"2023-06-23T18:03:37.768355+0000","last_became_peered":"2023-06-23T18:03:37.768355+0000","last_unstale":"2023-06-23T18:03:42.919917+0000","last_undegraded":"2023-06-23T18:03:42.919917+0000","last_fullsized":"2023-06-23T18:03:42.919917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919527+0000","last_change":"2023-06-23T18:03:37.764486+0000","last_active":"2023-06-23T18:03:42.919527+0000","last_peered":"2023-06-23T18:03:42.919527+0000","last_clean":"2023-06-23T18:03:42.919527+0000","last_became_active":"2023-06-23T18:03:37.755419+0000","last_became_peered":"2023-06-23T18:03:37.755419+0000","last_unstale":"2023-06-23T18:03:42.919527+0000","last_undegraded":"2023-06-23T18:03:42.919527+0000","last_fullsized":"2023-06-23T18:03:42.919527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921001+0000","last_change":"2023-06-23T18:03:37.770648+0000","last_active":"2023-06-23T18:03:42.921001+0000","last_peered":"2023-06-23T18:03:42.921001+0000","last_clean":"2023-06-23T18:03:42.921001+0000","last_became_active":"2023-06-23T18:03:37.770271+0000","last_became_peered":"2023-06-23T18:03:37.770271+0000","last_unstale":"2023-06-23T18:03:42.921001+0000","last_undegraded":"2023-06-23T18:03:42.921001+0000","last_fullsized":"2023-06-23T18:03:42.921001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916514+0000","last_change":"2023-06-23T18:03:37.767363+0000","last_active":"2023-06-23T18:03:42.916514+0000","last_peered":"2023-06-23T18:03:42.916514+0000","last_clean":"2023-06-23T18:03:42.916514+0000","last_became_active":"2023-06-23T18:03:37.766918+0000","last_became_peered":"2023-06-23T18:03:37.766918+0000","last_unstale":"2023-06-23T18:03:42.916514+0000","last_undegraded":"2023-06-23T18:03:42.916514+0000","last_fullsized":"2023-06-23T18:03:42.916514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:31.121336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920725+0000","last_change":"2023-06-23T18:03:37.757526+0000","last_active":"2023-06-23T18:03:42.920725+0000","last_peered":"2023-06-23T18:03:42.920725+0000","last_clean":"2023-06-23T18:03:42.920725+0000","last_became_active":"2023-06-23T18:03:37.757223+0000","last_became_peered":"2023-06-23T18:03:37.757223+0000","last_unstale":"2023-06-23T18:03:42.920725+0000","last_undegraded":"2023-06-23T18:03:42.920725+0000","last_fullsized":"2023-06-23T18:03:42.920725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921071+0000","last_change":"2023-06-23T18:03:37.770932+0000","last_active":"2023-06-23T18:03:42.921071+0000","last_peered":"2023-06-23T18:03:42.921071+0000","last_clean":"2023-06-23T18:03:42.921071+0000","last_became_active":"2023-06-23T18:03:37.770633+0000","last_became_peered":"2023-06-23T18:03:37.770633+0000","last_unstale":"2023-06-23T18:03:42.921071+0000","last_undegraded":"2023-06-23T18:03:42.921071+0000","last_fullsized":"2023-06-23T18:03:42.921071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158738+0000","last_change":"2023-06-23T18:03:37.757510+0000","last_active":"2023-06-23T18:03:42.158738+0000","last_peered":"2023-06-23T18:03:42.158738+0000","last_clean":"2023-06-23T18:03:42.158738+0000","last_became_active":"2023-06-23T18:03:37.757203+0000","last_became_peered":"2023-06-23T18:03:37.757203+0000","last_unstale":"2023-06-23T18:03:42.158738+0000","last_undegraded":"2023-06-23T18:03:42.158738+0000","last_fullsized":"2023-06-23T18:03:42.158738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:25:37.629512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153090+0000","last_change":"2023-06-23T18:03:37.760931+0000","last_active":"2023-06-23T18:03:42.153090+0000","last_peered":"2023-06-23T18:03:42.153090+0000","last_clean":"2023-06-23T18:03:42.153090+0000","last_became_active":"2023-06-23T18:03:37.760672+0000","last_became_peered":"2023-06-23T18:03:37.760672+0000","last_unstale":"2023-06-23T18:03:42.153090+0000","last_undegraded":"2023-06-23T18:03:42.153090+0000","last_fullsized":"2023-06-23T18:03:42.153090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:35.350379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158558+0000","last_change":"2023-06-23T18:03:37.761315+0000","last_active":"2023-06-23T18:03:42.158558+0000","last_peered":"2023-06-23T18:03:42.158558+0000","last_clean":"2023-06-23T18:03:42.158558+0000","last_became_active":"2023-06-23T18:03:37.760975+0000","last_became_peered":"2023-06-23T18:03:37.760975+0000","last_unstale":"2023-06-23T18:03:42.158558+0000","last_undegraded":"2023-06-23T18:03:42.158558+0000","last_fullsized":"2023-06-23T18:03:42.158558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:04.033803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922192+0000","last_change":"2023-06-23T18:03:37.754659+0000","last_active":"2023-06-23T18:03:42.922192+0000","last_peered":"2023-06-23T18:03:42.922192+0000","last_clean":"2023-06-23T18:03:42.922192+0000","last_became_active":"2023-06-23T18:03:37.751277+0000","last_became_peered":"2023-06-23T18:03:37.751277+0000","last_unstale":"2023-06-23T18:03:42.922192+0000","last_undegraded":"2023-06-23T18:03:42.922192+0000","last_fullsized":"2023-06-23T18:03:42.922192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156785+0000","last_change":"2023-06-23T18:03:37.752494+0000","last_active":"2023-06-23T18:03:42.156785+0000","last_peered":"2023-06-23T18:03:42.156785+0000","last_clean":"2023-06-23T18:03:42.156785+0000","last_became_active":"2023-06-23T18:03:37.752149+0000","last_became_peered":"2023-06-23T18:03:37.752149+0000","last_unstale":"2023-06-23T18:03:42.156785+0000","last_undegraded":"2023-06-23T18:03:42.156785+0000","last_fullsized":"2023-06-23T18:03:42.156785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:37.654382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921056+0000","last_change":"2023-06-23T18:03:37.775997+0000","last_active":"2023-06-23T18:03:42.921056+0000","last_peered":"2023-06-23T18:03:42.921056+0000","last_clean":"2023-06-23T18:03:42.921056+0000","last_became_active":"2023-06-23T18:03:37.775685+0000","last_became_peered":"2023-06-23T18:03:37.775685+0000","last_unstale":"2023-06-23T18:03:42.921056+0000","last_undegraded":"2023-06-23T18:03:42.921056+0000","last_fullsized":"2023-06-23T18:03:42.921056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923426+0000","last_change":"2023-06-23T18:03:37.763367+0000","last_active":"2023-06-23T18:03:42.923426+0000","last_peered":"2023-06-23T18:03:42.923426+0000","last_clean":"2023-06-23T18:03:42.923426+0000","last_became_active":"2023-06-23T18:03:37.763035+0000","last_became_peered":"2023-06-23T18:03:37.763035+0000","last_unstale":"2023-06-23T18:03:42.923426+0000","last_undegraded":"2023-06-23T18:03:42.923426+0000","last_fullsized":"2023-06-23T18:03:42.923426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152175+0000","last_change":"2023-06-23T18:03:37.758554+0000","last_active":"2023-06-23T18:03:42.152175+0000","last_peered":"2023-06-23T18:03:42.152175+0000","last_clean":"2023-06-23T18:03:42.152175+0000","last_became_active":"2023-06-23T18:03:37.758307+0000","last_became_peered":"2023-06-23T18:03:37.758307+0000","last_unstale":"2023-06-23T18:03:42.152175+0000","last_undegraded":"2023-06-23T18:03:42.152175+0000","last_fullsized":"2023-06-23T18:03:42.152175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:23:09.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152731+0000","last_change":"2023-06-23T18:03:37.760705+0000","last_active":"2023-06-23T18:03:42.152731+0000","last_peered":"2023-06-23T18:03:42.152731+0000","last_clean":"2023-06-23T18:03:42.152731+0000","last_became_active":"2023-06-23T18:03:37.759846+0000","last_became_peered":"2023-06-23T18:03:37.759846+0000","last_unstale":"2023-06-23T18:03:42.152731+0000","last_undegraded":"2023-06-23T18:03:42.152731+0000","last_fullsized":"2023-06-23T18:03:42.152731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:26.801591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158644+0000","last_change":"2023-06-23T18:03:37.757927+0000","last_active":"2023-06-23T18:03:42.158644+0000","last_peered":"2023-06-23T18:03:42.158644+0000","last_clean":"2023-06-23T18:03:42.158644+0000","last_became_active":"2023-06-23T18:03:37.757635+0000","last_became_peered":"2023-06-23T18:03:37.757635+0000","last_unstale":"2023-06-23T18:03:42.158644+0000","last_undegraded":"2023-06-23T18:03:42.158644+0000","last_fullsized":"2023-06-23T18:03:42.158644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:26.427732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923450+0000","last_change":"2023-06-23T18:03:37.768282+0000","last_active":"2023-06-23T18:03:42.923450+0000","last_peered":"2023-06-23T18:03:42.923450+0000","last_clean":"2023-06-23T18:03:42.923450+0000","last_became_active":"2023-06-23T18:03:37.768038+0000","last_became_peered":"2023-06-23T18:03:37.768038+0000","last_unstale":"2023-06-23T18:03:42.923450+0000","last_undegraded":"2023-06-23T18:03:42.923450+0000","last_fullsized":"2023-06-23T18:03:42.923450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922018+0000","last_change":"2023-06-23T18:03:37.752981+0000","last_active":"2023-06-23T18:03:42.922018+0000","last_peered":"2023-06-23T18:03:42.922018+0000","last_clean":"2023-06-23T18:03:42.922018+0000","last_became_active":"2023-06-23T18:03:37.752291+0000","last_became_peered":"2023-06-23T18:03:37.752291+0000","last_unstale":"2023-06-23T18:03:42.922018+0000","last_undegraded":"2023-06-23T18:03:42.922018+0000","last_fullsized":"2023-06-23T18:03:42.922018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919440+0000","last_change":"2023-06-23T18:03:37.756558+0000","last_active":"2023-06-23T18:03:42.919440+0000","last_peered":"2023-06-23T18:03:42.919440+0000","last_clean":"2023-06-23T18:03:42.919440+0000","last_became_active":"2023-06-23T18:03:37.756044+0000","last_became_peered":"2023-06-23T18:03:37.756044+0000","last_unstale":"2023-06-23T18:03:42.919440+0000","last_undegraded":"2023-06-23T18:03:42.919440+0000","last_fullsized":"2023-06-23T18:03:42.919440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920901+0000","last_change":"2023-06-23T18:03:37.775992+0000","last_active":"2023-06-23T18:03:42.920901+0000","last_peered":"2023-06-23T18:03:42.920901+0000","last_clean":"2023-06-23T18:03:42.920901+0000","last_became_active":"2023-06-23T18:03:37.775667+0000","last_became_peered":"2023-06-23T18:03:37.775667+0000","last_unstale":"2023-06-23T18:03:42.920901+0000","last_undegraded":"2023-06-23T18:03:42.920901+0000","last_fullsized":"2023-06-23T18:03:42.920901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157021+0000","last_change":"2023-06-23T18:03:37.752811+0000","last_active":"2023-06-23T18:03:42.157021+0000","last_peered":"2023-06-23T18:03:42.157021+0000","last_clean":"2023-06-23T18:03:42.157021+0000","last_became_active":"2023-06-23T18:03:37.752518+0000","last_became_peered":"2023-06-23T18:03:37.752518+0000","last_unstale":"2023-06-23T18:03:42.157021+0000","last_undegraded":"2023-06-23T18:03:42.157021+0000","last_fullsized":"2023-06-23T18:03:42.157021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:26.870651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920914+0000","last_change":"2023-06-23T18:03:37.757769+0000","last_active":"2023-06-23T18:03:42.920914+0000","last_peered":"2023-06-23T18:03:42.920914+0000","last_clean":"2023-06-23T18:03:42.920914+0000","last_became_active":"2023-06-23T18:03:37.757520+0000","last_became_peered":"2023-06-23T18:03:37.757520+0000","last_unstale":"2023-06-23T18:03:42.920914+0000","last_undegraded":"2023-06-23T18:03:42.920914+0000","last_fullsized":"2023-06-23T18:03:42.920914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919630+0000","last_change":"2023-06-23T18:03:37.755865+0000","last_active":"2023-06-23T18:03:42.919630+0000","last_peered":"2023-06-23T18:03:42.919630+0000","last_clean":"2023-06-23T18:03:42.919630+0000","last_became_active":"2023-06-23T18:03:37.755153+0000","last_became_peered":"2023-06-23T18:03:37.755153+0000","last_unstale":"2023-06-23T18:03:42.919630+0000","last_undegraded":"2023-06-23T18:03:42.919630+0000","last_fullsized":"2023-06-23T18:03:42.919630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:42:12.603650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158475+0000","last_change":"2023-06-23T18:03:37.754864+0000","last_active":"2023-06-23T18:03:42.158475+0000","last_peered":"2023-06-23T18:03:42.158475+0000","last_clean":"2023-06-23T18:03:42.158475+0000","last_became_active":"2023-06-23T18:03:37.754386+0000","last_became_peered":"2023-06-23T18:03:37.754386+0000","last_unstale":"2023-06-23T18:03:42.158475+0000","last_undegraded":"2023-06-23T18:03:42.158475+0000","last_fullsized":"2023-06-23T18:03:42.158475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:30:06.917401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922699+0000","last_change":"2023-06-23T18:03:37.757620+0000","last_active":"2023-06-23T18:03:42.922699+0000","last_peered":"2023-06-23T18:03:42.922699+0000","last_clean":"2023-06-23T18:03:42.922699+0000","last_became_active":"2023-06-23T18:03:37.757346+0000","last_became_peered":"2023-06-23T18:03:37.757346+0000","last_unstale":"2023-06-23T18:03:42.922699+0000","last_undegraded":"2023-06-23T18:03:42.922699+0000","last_fullsized":"2023-06-23T18:03:42.922699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:59.832808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152893+0000","last_change":"2023-06-23T18:03:37.759638+0000","last_active":"2023-06-23T18:03:42.152893+0000","last_peered":"2023-06-23T18:03:42.152893+0000","last_clean":"2023-06-23T18:03:42.152893+0000","last_became_active":"2023-06-23T18:03:37.759380+0000","last_became_peered":"2023-06-23T18:03:37.759380+0000","last_unstale":"2023-06-23T18:03:42.152893+0000","last_undegraded":"2023-06-23T18:03:42.152893+0000","last_fullsized":"2023-06-23T18:03:42.152893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:44.739308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31300,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340540,"statfs":{"total":96636764160,"available":96604712960,"internally_reserved":0,"allocated":258048,"data_stored":104812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":851968,"data_stored":695672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:04:02.110 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-23T18:04:02.110 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-23T18:04:02.111 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-23T18:04:02.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-23T18:04:02.111 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-23T18:04:02.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 Processor -- start 2023-06-23T18:04:02.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 -- start start 2023-06-23T18:04:02.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14d4069960 0x7f14d40cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 0x7f14d406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d40c55f0 con 0x7f14d415bd80 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d406b280 con 0x7f14d4069960 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d8841700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40c4550 con 0x7f14d40cc8a0 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 0x7f14d406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d2d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14d4069960 0x7f14d40cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d259c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f14d415bd80 0x7f14d40c8ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59822/0 (socket says 172.21.15.22:59822) 2023-06-23T18:04:02.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d259c700 1 -- 172.21.15.22:0/1334086671 learned_addr learned my addr 172.21.15.22:0/1334086671 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:02.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d359e700 1 -- 172.21.15.22:0/1334086671 >> v2:172.21.15.169:3300/0 conn(0x7f14d4069960 msgr2=0x7f14d40cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:02.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d1d9b700 1 -- 172.21.15.22:0/1334086671 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307680272 0 0) 0x7f14d40c55f0 con 0x7f14d415bd80 2023-06-23T18:04:02.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d359e700 1 --2- 172.21.15.22:0/1334086671 >> v2:172.21.15.169:3300/0 conn(0x7f14d4069960 0x7f14d40cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d359e700 1 -- 172.21.15.22:0/1334086671 >> v1:172.21.15.169:6789/0 conn(0x7f14d415bd80 legacy=0x7f14d40c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.215+0000 7f14d359e700 1 -- 172.21.15.22:0/1334086671 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40c4830 con 0x7f14d40cc8a0 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d359e700 1 --2- 172.21.15.22:0/1334086671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 0x7f14d406ad40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009d90 tx=0x7f14c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1abd33d68c389ff server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d1d9b700 1 -- 172.21.15.22:0/1334086671 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400e070 con 0x7f14d40cc8a0 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d1d9b700 1 -- 172.21.15.22:0/1334086671 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c400bb20 con 0x7f14d40cc8a0 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d1d9b700 1 -- 172.21.15.22:0/1334086671 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4012430 con 0x7f14d40cc8a0 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- 172.21.15.22:0/1334086671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 msgr2=0x7f14d406ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- 172.21.15.22:0/1334086671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 0x7f14d406ad40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009d90 tx=0x7f14c400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- 172.21.15.22:0/1334086671 shutdown_connections 2023-06-23T18:04:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- 172.21.15.22:0/1334086671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d40cc8a0 0x7f14d406ad40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- 172.21.15.22:0/1334086671 >> v2:172.21.15.169:3300/0 conn(0x7f14d4069960 0x7f14d40cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- 172.21.15.22:0/1334086671 >> 172.21.15.22:0/1334086671 conn(0x7f14d40cb2a0 msgr2=0x7f14d40cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- 172.21.15.22:0/1334086671 shutdown_connections 2023-06-23T18:04:02.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- 172.21.15.22:0/1334086671 wait complete. 2023-06-23T18:04:02.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 Processor -- start 2023-06-23T18:04:02.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- start start 2023-06-23T18:04:02.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d4069960 0x7f14d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 0x7f14d4070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d414a160 con 0x7f14d40b1770 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40c5990 con 0x7f14d4069960 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d8841700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40715f0 con 0x7f14d415bd80 2023-06-23T18:04:02.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43834/0 (socket says 172.21.15.22:43834) 2023-06-23T18:04:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.219+0000 7f14d359e700 1 -- 172.21.15.22:0/2884977559 learned_addr learned my addr 172.21.15.22:0/2884977559 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d259c700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 0x7f14d4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d2d9d700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d4069960 0x7f14d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 -- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 msgr2=0x7f14d4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 0x7f14d4070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 -- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d4069960 msgr2=0x7f14d40710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d4069960 0x7f14d40710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 -- 172.21.15.22:0/2884977559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4009a70 con 0x7f14d40b1770 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d259c700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 0x7f14d4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d359e700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14c4013000 tx=0x7f14c4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4012430 con 0x7f14d40b1770 2023-06-23T18:04:02.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4020070 con 0x7f14d40b1770 2023-06-23T18:04:02.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4004280 con 0x7f14d40b1770 2023-06-23T18:04:02.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40b1be0 con 0x7f14d40b1770 2023-06-23T18:04:02.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40b21a0 con 0x7f14d40b1770 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d40c5b90 con 0x7f14d40b1770 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.223+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f14c40043e0 con 0x7f14d40b1770 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.227+0000 7f14cb7fe700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 0x7f14b80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.227+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14c4022070 con 0x7f14d40b1770 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.227+0000 7f14d2d9d700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 0x7f14b80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.227+0000 7f14d2d9d700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 0x7f14b80687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ae0 tx=0x7f14cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.231+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f14d40c5b90 con 0x7f14d40b1770 2023-06-23T18:04:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.439+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f14d40c5b50 con 0x7f14d40b1770 2023-06-23T18:04:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.443+0000 7f14cb7fe700 1 -- 172.21.15.22:0/2884977559 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164241 (secure 0 0 0) 0x7f14c4052ec0 con 0x7f14d40b1770 2023-06-23T18:04:02.463 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:02.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.447+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 msgr2=0x7f14b80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.447+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 0x7f14b80687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ae0 tx=0x7f14cc008040 comp rx=0 tx=0).stop 2023-06-23T18:04:02.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.447+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 msgr2=0x7f14d41f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.447+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14c4013000 tx=0x7f14c4004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 shutdown_connections 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14d40b1770 0x7f14d41f06e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d415bd80 0x7f14d4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f14b80662f0 0x7f14b80687a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 --2- 172.21.15.22:0/2884977559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d4069960 0x7f14d40710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 >> 172.21.15.22:0/2884977559 conn(0x7f14d40cb2a0 msgr2=0x7f14d406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 shutdown_connections 2023-06-23T18:04:02.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.451+0000 7f14d8841700 1 -- 172.21.15.22:0/2884977559 wait complete. 2023-06-23T18:04:02.478 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3032843375},{"type":"v1","addr":"172.21.15.22:6833","nonce":3032843375}]},"active_addr":"172.21.15.22:6833/3032843375","active_change":"2023-06-23T18:03:25.589653+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2113331782}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":718509239}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2205251571}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2366932129}]}]} 2023-06-23T18:04:02.483 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-23T18:04:02.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-23T18:04:02.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 Processor -- start 2023-06-23T18:04:02.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 -- start start 2023-06-23T18:04:02.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff12806c610 0x7ff1280c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280c9b50 con 0x7ff1280b01a0 2023-06-23T18:04:02.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280c9e30 con 0x7ff128070280 2023-06-23T18:04:02.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff127fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1281590a0 con 0x7ff12806c610 2023-06-23T18:04:02.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff1267fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff1280b01a0 0x7ff1280c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37106/0 (socket says 172.21.15.22:37106) 2023-06-23T18:04:02.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff1267fc700 1 -- 172.21.15.22:0/4114367079 learned_addr learned my addr 172.21.15.22:0/4114367079 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133289228 0 0) 0x7ff1280c9b50 con 0x7ff1280b01a0 2023-06-23T18:04:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff1277fe700 1 --2- 172.21.15.22:0/4114367079 >> v2:172.21.15.169:3300/0 conn(0x7ff12806c610 0x7ff1280c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1100035d0 con 0x7ff1280b01a0 2023-06-23T18:04:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 338961940 0 0) 0x7ff1100035d0 con 0x7ff1280b01a0 2023-06-23T18:04:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 >> v2:172.21.15.169:3300/0 conn(0x7ff12806c610 msgr2=0x7ff1280c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 --2- 172.21.15.22:0/4114367079 >> v2:172.21.15.169:3300/0 conn(0x7ff12806c610 0x7ff1280c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 >> v1:172.21.15.169:6789/0 conn(0x7ff128070280 legacy=0x7ff1280afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280ca110 con 0x7ff1280b01a0 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7ff1180019f0 con 0x7ff1280b01a0 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff1277fe700 1 --2- 172.21.15.22:0/4114367079 >> v2:172.21.15.169:3300/0 conn(0x7ff12806c610 0x7ff1280c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 >> v1:172.21.15.22:6789/0 conn(0x7ff1280b01a0 legacy=0x7ff1280c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110003a20 0x7ff110003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 0x7ff110004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff110005220 0x7ff11000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281590a0 con 0x7ff110004870 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11000c190 con 0x7ff110005220 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11000c350 con 0x7ff110003a20 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.583+0000 7ff1267fc700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 0x7ff110004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1277fe700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff110005220 0x7ff11000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff126ffd700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110003a20 0x7ff110003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 -- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110003a20 msgr2=0x7ff110003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110003a20 0x7ff110003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 -- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff110005220 msgr2=0x7ff11000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff110005220 0x7ff11000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 -- 172.21.15.22:0/4114367079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100035d0 con 0x7ff110004870 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1277fe700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff110005220 0x7ff11000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1277fe700 1 -- 172.21.15.22:0/4114367079 reap_dead start 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff1267fc700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 0x7ff110004ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a830 tx=0x7ff1100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3c71b98dad995e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800a2f0 con 0x7ff110004870 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1180072f0 con 0x7ff110004870 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff125ffb700 1 -- 172.21.15.22:0/4114367079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180076a0 con 0x7ff110004870 2023-06-23T18:04:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 msgr2=0x7ff110004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 0x7ff110004ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a830 tx=0x7ff1100058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- 172.21.15.22:0/4114367079 shutdown_connections 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 --2- 172.21.15.22:0/4114367079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff110004870 0x7ff110004ce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- 172.21.15.22:0/4114367079 >> 172.21.15.22:0/4114367079 conn(0x7ff12806a5f0 msgr2=0x7ff1280caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- 172.21.15.22:0/4114367079 shutdown_connections 2023-06-23T18:04:02.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- 172.21.15.22:0/4114367079 wait complete. 2023-06-23T18:04:02.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 Processor -- start 2023-06-23T18:04:02.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.587+0000 7ff127fff700 1 -- start start 2023-06-23T18:04:02.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110004870 0x7ff1281591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1281f47f0 0x7ff1280c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280cbdf0 con 0x7ff1280c6670 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128159720 con 0x7ff1281f47f0 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff128159980 con 0x7ff110004870 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43852/0 (socket says 172.21.15.22:43852) 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 -- 172.21.15.22:0/2864767755 learned_addr learned my addr 172.21.15.22:0/2864767755 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110004870 0x7ff1281591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1267fc700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1281f47f0 0x7ff1280c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 -- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110004870 msgr2=0x7ff1281591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110004870 0x7ff1281591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 -- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1281f47f0 msgr2=0x7ff1280c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1281f47f0 0x7ff1280c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 -- 172.21.15.22:0/2864767755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128159c60 con 0x7ff1280c6670 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff1277fe700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1280c85f0 tx=0x7ff120010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120004020 con 0x7ff1280c6670 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000c050 con 0x7ff1280c6670 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200043a0 con 0x7ff1280c6670 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff128159f40 con 0x7ff1280c6670 2023-06-23T18:04:02.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.591+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12815ad70 con 0x7ff1280c6670 2023-06-23T18:04:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12815aff0 con 0x7ff1280c6670 2023-06-23T18:04:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff1200036c0 con 0x7ff1280c6670 2023-06-23T18:04:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff10f7fe700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 0x7ff1140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff120083800 con 0x7ff1280c6670 2023-06-23T18:04:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 0x7ff1140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.595+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 0x7ff1140687a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.603+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff120050a20 con 0x7ff1280c6670 2023-06-23T18:04:02.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.771+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1280cbbe0 con 0x7ff1280c6670 2023-06-23T18:04:02.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.771+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2864767755 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7ff120054040 con 0x7ff1280c6670 2023-06-23T18:04:02.788 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:02.789 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:02.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 msgr2=0x7ff1140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 0x7ff1140687a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 msgr2=0x7ff12815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1280c85f0 tx=0x7ff120010840 comp rx=0 tx=0).stop 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 shutdown_connections 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280c6670 0x7ff12815a650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff1140662f0 0x7ff1140687a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1281f47f0 0x7ff1280c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 --2- 172.21.15.22:0/2864767755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff110004870 0x7ff1281591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 >> 172.21.15.22:0/2864767755 conn(0x7ff12806a5f0 msgr2=0x7ff1281585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 shutdown_connections 2023-06-23T18:04:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.775+0000 7ff127fff700 1 -- 172.21.15.22:0/2864767755 wait complete. 2023-06-23T18:04:02.804 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-23T18:04:02.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 Processor -- start 2023-06-23T18:04:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 -- start start 2023-06-23T18:04:02.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1bd015beb0 0x7f1bd0070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1bd015c280 0x7f1bd00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd00ca370 con 0x7f1bd006d110 2023-06-23T18:04:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd00710e0 con 0x7f1bd015beb0 2023-06-23T18:04:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5d57700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd00c8790 con 0x7f1bd015c280 2023-06-23T18:04:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1bd015beb0 0x7f1bd0070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5556700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1bd006d110 0x7f1bd015fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37110/0 (socket says 172.21.15.22:37110) 2023-06-23T18:04:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1bd015beb0 0x7f1bd0070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43860/0 (socket says 172.21.15.22:43860) 2023-06-23T18:04:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd5556700 1 -- 172.21.15.22:0/2068126518 learned_addr learned my addr 172.21.15.22:0/2068126518 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bcffff700 1 --2- 172.21.15.22:0/2068126518 >> v2:172.21.15.169:3300/0 conn(0x7f1bd015c280 0x7f1bd00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 -- 172.21.15.22:0/2068126518 >> v2:172.21.15.169:3300/0 conn(0x7f1bd015c280 msgr2=0x7f1bd00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 --2- 172.21.15.22:0/2068126518 >> v2:172.21.15.169:3300/0 conn(0x7f1bd015c280 0x7f1bd00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 -- 172.21.15.22:0/2068126518 >> v1:172.21.15.22:6789/0 conn(0x7f1bd006d110 legacy=0x7f1bd015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bcf7fe700 1 -- 172.21.15.22:0/2068126518 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116394274 0 0) 0x7f1bd00ca370 con 0x7f1bd006d110 2023-06-23T18:04:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 -- 172.21.15.22:0/2068126518 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0160780 con 0x7f1bd015beb0 2023-06-23T18:04:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bd4d55700 1 --2- 172.21.15.22:0/2068126518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd015beb0 0x7f1bd0070ba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0160970 tx=0x7f1bb8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33da677240123716 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bcf7fe700 1 -- 172.21.15.22:0/2068126518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8004020 con 0x7f1bd015beb0 2023-06-23T18:04:02.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bcf7fe700 1 -- 172.21.15.22:0/2068126518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8004180 con 0x7f1bd015beb0 2023-06-23T18:04:02.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.903+0000 7f1bcf7fe700 1 -- 172.21.15.22:0/2068126518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8004530 con 0x7f1bd015beb0 2023-06-23T18:04:02.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- 172.21.15.22:0/2068126518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd015beb0 msgr2=0x7f1bd0070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/2068126518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd015beb0 0x7f1bd0070ba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0160970 tx=0x7f1bb8009b30 comp rx=0 tx=0).stop 2023-06-23T18:04:02.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- 172.21.15.22:0/2068126518 shutdown_connections 2023-06-23T18:04:02.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/2068126518 >> v2:172.21.15.169:3300/0 conn(0x7f1bd015c280 0x7f1bd00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/2068126518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd015beb0 0x7f1bd0070ba0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- 172.21.15.22:0/2068126518 >> 172.21.15.22:0/2068126518 conn(0x7f1bd006a5f0 msgr2=0x7f1bd00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- 172.21.15.22:0/2068126518 shutdown_connections 2023-06-23T18:04:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- 172.21.15.22:0/2068126518 wait complete. 2023-06-23T18:04:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 Processor -- start 2023-06-23T18:04:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.907+0000 7f1bd5d57700 1 -- start start 2023-06-23T18:04:02.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd015beb0 0x7f1bd00b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd015c280 0x7f1bd01f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00b2d70 con 0x7f1bd006d110 2023-06-23T18:04:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00b3bd0 con 0x7f1bd015c280 2023-06-23T18:04:02.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00b23f0 con 0x7f1bd015beb0 2023-06-23T18:04:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43876/0 (socket says 172.21.15.22:43876) 2023-06-23T18:04:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 -- 172.21.15.22:0/3616888456 learned_addr learned my addr 172.21.15.22:0/3616888456 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:02.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5556700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd015c280 0x7f1bd01f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bcffff700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd015beb0 0x7f1bd00b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 -- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd015beb0 msgr2=0x7f1bd00b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd015beb0 0x7f1bd00b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 -- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd015c280 msgr2=0x7f1bd01f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:02.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd015c280 0x7f1bd01f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:02.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 -- 172.21.15.22:0/3616888456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80094b0 con 0x7f1bd006d110 2023-06-23T18:04:02.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd4d55700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8011cb0 con 0x7f1bd006d110 2023-06-23T18:04:02.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8011e30 con 0x7f1bd006d110 2023-06-23T18:04:02.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd00b2fc0 con 0x7f1bd006d110 2023-06-23T18:04:02.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8003900 con 0x7f1bd006d110 2023-06-23T18:04:02.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd00b3580 con 0x7f1bd006d110 2023-06-23T18:04:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.911+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd004e4c0 con 0x7f1bd006d110 2023-06-23T18:04:02.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.915+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1bb8003a60 con 0x7f1bd006d110 2023-06-23T18:04:02.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.915+0000 7f1bcd7fa700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 0x7f1bb40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:02.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.915+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1bb8084f20 con 0x7f1bd006d110 2023-06-23T18:04:02.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.915+0000 7f1bcffff700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 0x7f1bb40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:02.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.915+0000 7f1bcffff700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 0x7f1bb40687a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00c9580 tx=0x7f1bc800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:02.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:02.923+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1bb8052140 con 0x7f1bd006d110 2023-06-23T18:04:03.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.087+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bd00c9b70 con 0x7f1bd006d110 2023-06-23T18:04:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bcd7fa700 1 -- 172.21.15.22:0/3616888456 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f1bb8055760 con 0x7f1bd006d110 2023-06-23T18:04:03.106 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:03.106 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 msgr2=0x7f1bb40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 0x7f1bb40687a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00c9580 tx=0x7f1bc800b040 comp rx=0 tx=0).stop 2023-06-23T18:04:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 msgr2=0x7f1bd00b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb8004770 comp rx=0 tx=0).stop 2023-06-23T18:04:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 shutdown_connections 2023-06-23T18:04:03.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1bb40662f0 0x7f1bb40687a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd015c280 0x7f1bd01f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd015beb0 0x7f1bd00b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 --2- 172.21.15.22:0/3616888456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bd006d110 0x7f1bd00b1490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 >> 172.21.15.22:0/3616888456 conn(0x7f1bd006a5f0 msgr2=0x7f1bd00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 shutdown_connections 2023-06-23T18:04:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.091+0000 7f1bd5d57700 1 -- 172.21.15.22:0/3616888456 wait complete. 2023-06-23T18:04:03.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-23T18:04:03.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-23T18:04:03.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-23T18:04:03.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-23T18:04:03.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-23T18:04:03.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-23T18:04:03.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-23T18:04:03.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-23T18:04:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 Processor -- start 2023-06-23T18:04:03.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 -- start start 2023-06-23T18:04:03.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 Processor -- start 2023-06-23T18:04:03.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 Processor -- start 2023-06-23T18:04:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 -- start start 2023-06-23T18:04:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 -- start start 2023-06-23T18:04:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 Processor -- start 2023-06-23T18:04:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 Processor -- start 2023-06-23T18:04:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 -- start start 2023-06-23T18:04:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 -- start start 2023-06-23T18:04:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 0x7f7c8015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c800afa90 0x7f7c800aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800ca010 con 0x7f7c80071660 2023-06-23T18:04:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800b0440 con 0x7f7c800afa90 2023-06-23T18:04:03.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801582c0 con 0x7f7c8006f890 2023-06-23T18:04:03.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f26e80cbbe0 0x7f26e80713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e8071930 con 0x7f26e80cd310 2023-06-23T18:04:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e80cc000 con 0x7f26e8158050 2023-06-23T18:04:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26eca51700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80b0210 con 0x7f26e80cbbe0 2023-06-23T18:04:03.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd020 0x7f6dd00c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6dd00cd4e0 0x7f6dd00cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00cde80 con 0x7f6dd0071500 2023-06-23T18:04:03.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd015c270 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd6174700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00b02e0 con 0x7f6dd00cd020 2023-06-23T18:04:03.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6dd00cd4e0 0x7f6dd00cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6dd00cd4e0 0x7f6dd00cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43890/0 (socket says 172.21.15.22:43890) 2023-06-23T18:04:03.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 -- 172.21.15.22:0/4083702409 learned_addr learned my addr 172.21.15.22:0/4083702409 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe31c0ad380 0x7fe31c0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0ce200 con 0x7fe31c0c9d00 2023-06-23T18:04:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0ceb10 con 0x7fe31c0ad380 2023-06-23T18:04:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe321686700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ad8d0 con 0x7fe31c0cb130 2023-06-23T18:04:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fb2f33cf700 1 Processor -- start 2023-06-23T18:04:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fb2f33cf700 1 -- start start 2023-06-23T18:04:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26e6d9d700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f26e8158050 0x7f26e80afb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59860/0 (socket says 172.21.15.22:59860) 2023-06-23T18:04:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26e6d9d700 1 -- 172.21.15.22:0/3561443184 learned_addr learned my addr 172.21.15.22:0/3561443184 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 -- 172.21.15.22:0/4083702409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd020 msgr2=0x7f6dd00c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 --2- 172.21.15.22:0/4083702409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd020 0x7f6dd00c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 -- 172.21.15.22:0/4083702409 >> v1:172.21.15.169:6789/0 conn(0x7f6dd0071500 legacy=0x7f6dd00afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe31c0ad380 0x7fe31c0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe31c0ad380 0x7fe31c0abe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43902/0 (socket says 172.21.15.22:43902) 2023-06-23T18:04:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 -- 172.21.15.22:0/2256606916 learned_addr learned my addr 172.21.15.22:0/2256606916 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31a7fc700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2c500cc880 0x7f2c5006d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2c500afa90 0x7f2c500afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500b03d0 con 0x7f2c500ccc50 2023-06-23T18:04:03.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c501582c0 con 0x7f2c500afa90 2023-06-23T18:04:03.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f6dd5973700 1 -- 172.21.15.22:0/4083702409 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00c8800 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4ffff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5006d540 con 0x7f2c500cc880 2023-06-23T18:04:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2c500afa90 0x7f2c500afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2c500afa90 0x7f2c500afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43920/0 (socket says 172.21.15.22:43920) 2023-06-23T18:04:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f2c4f7fe700 1 -- 172.21.15.22:0/3709204597 learned_addr learned my addr 172.21.15.22:0/3709204597 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f26e759e700 1 --2- 172.21.15.22:0/3561443184 >> v2:172.21.15.22:3300/0 conn(0x7f26e80cbbe0 0x7f26e80713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 -- 172.21.15.22:0/2256606916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 msgr2=0x7fe31c0ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 -- 172.21.15.22:0/2256606916 >> v1:172.21.15.22:6789/0 conn(0x7fe31c0c9d00 legacy=0x7fe31c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7fe31b7fe700 1 -- 172.21.15.22:0/2256606916 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0ce450 con 0x7fe31c0ad380 2023-06-23T18:04:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4effd700 1 --2- 172.21.15.22:0/3709204597 >> v2:172.21.15.169:3300/0 conn(0x7f2c500cc880 0x7f2c5006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f10c9754700 1 Processor -- start 2023-06-23T18:04:03.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f10c9754700 1 -- start start 2023-06-23T18:04:03.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 Processor -- start 2023-06-23T18:04:03.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4f7fe700 1 -- 172.21.15.22:0/3709204597 >> v2:172.21.15.169:3300/0 conn(0x7f2c500cc880 msgr2=0x7f2c5006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4f7fe700 1 --2- 172.21.15.22:0/3709204597 >> v2:172.21.15.169:3300/0 conn(0x7f2c500cc880 0x7f2c5006d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4f7fe700 1 -- 172.21.15.22:0/3709204597 >> v1:172.21.15.22:6789/0 conn(0x7f2c500ccc50 legacy=0x7f2c500cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4f7fe700 1 -- 172.21.15.22:0/3709204597 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50071920 con 0x7f2c500afa90 2023-06-23T18:04:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd5973700 1 --2- 172.21.15.22:0/4083702409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cd4e0 0x7f6dd00cd940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00ffd0 tx=0x7f6dcc014850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecd35256984f52e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe31b7fe700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c0abe30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe310009910 tx=0x7fe310009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c63c897c47f294e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 -- start start 2023-06-23T18:04:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 0x7f10c40d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c40ce350 con 0x7f10c40cdde0 2023-06-23T18:04:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c40ce630 con 0x7f10c40cf4b0 2023-06-23T18:04:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40d1960 con 0x7f10c415fee0 2023-06-23T18:04:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c800afa90 0x7f7c800aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7effd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7c80071660 0x7f7c8006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37134/0 (socket says 172.21.15.22:37134) 2023-06-23T18:04:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7effd700 1 -- 172.21.15.22:0/2382939755 learned_addr learned my addr 172.21.15.22:0/2382939755 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c800afa90 0x7f7c800aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43900/0 (socket says 172.21.15.22:43900) 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.263+0000 7f7c7e7fc700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 0x7f7c8015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/2382939755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 msgr2=0x7f7c8015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7f7fe700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 0x7f7c8015c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/2382939755 >> v1:172.21.15.22:6789/0 conn(0x7f7c80071660 legacy=0x7f7c8006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/2382939755 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8015bae0 con 0x7f7c800afa90 2023-06-23T18:04:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7e7fc700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 0x7f7c8015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7f7fe700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c800afa90 0x7f7c800aff00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009df0 tx=0x7f7c7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47a43b61c01711b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f33cf700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2ec0cf1b0 0x7fb2ec0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f33cf700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec0b5a70 0x7fb2ec0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f33cf700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec15a080 con 0x7fb2ec0cde70 2023-06-23T18:04:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f33cf700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b62e0 con 0x7fb2ec0b5a70 2023-06-23T18:04:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f33cf700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cc830 con 0x7fb2ec0cf1b0 2023-06-23T18:04:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7dffb700 1 -- 172.21.15.22:0/2382939755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800bd50 con 0x7f7c800afa90 2023-06-23T18:04:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4f7fe700 1 --2- 172.21.15.22:0/3709204597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500afa90 0x7f2c500afe90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c500cf110 tx=0x7f2c4800d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d0f9ca543decc3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f2bce700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec0b5a70 0x7fb2ec0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dc7fff700 1 -- 172.21.15.22:0/4083702409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc012660 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dc7fff700 1 -- 172.21.15.22:0/4083702409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc0127c0 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dc7fff700 1 -- 172.21.15.22:0/4083702409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc012b50 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c37fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f10c40cdde0 0x7f10c40ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37156/0 (socket says 172.21.15.22:37156) 2023-06-23T18:04:03.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c37fe700 1 -- 172.21.15.22:0/1466821133 learned_addr learned my addr 172.21.15.22:0/1466821133 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:03.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c3fff700 1 --2- 172.21.15.22:0/1466821133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 0x7f10c40d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f2bce700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec0b5a70 0x7fb2ec0cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43936/0 (socket says 172.21.15.22:43936) 2023-06-23T18:04:03.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f2bce700 1 -- 172.21.15.22:0/418787941 learned_addr learned my addr 172.21.15.22:0/418787941 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7dffb700 1 -- 172.21.15.22:0/2382939755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78003a40 con 0x7f7c800afa90 2023-06-23T18:04:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7dffb700 1 -- 172.21.15.22:0/2382939755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78003df0 con 0x7f7c800afa90 2023-06-23T18:04:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe319ffb700 1 -- 172.21.15.22:0/2256606916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310004230 con 0x7fe31c0ad380 2023-06-23T18:04:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe319ffb700 1 -- 172.21.15.22:0/2256606916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe310004390 con 0x7fe31c0ad380 2023-06-23T18:04:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe319ffb700 1 -- 172.21.15.22:0/2256606916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310009e60 con 0x7fe31c0ad380 2023-06-23T18:04:03.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e759e700 1 -- 172.21.15.22:0/3561443184 >> v1:172.21.15.169:6789/0 conn(0x7f26e8158050 legacy=0x7f26e80afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e759e700 1 -- 172.21.15.22:0/3561443184 >> v1:172.21.15.22:6789/0 conn(0x7f26e80cd310 legacy=0x7f26e80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e759e700 1 -- 172.21.15.22:0/3561443184 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8161160 con 0x7f26e80cbbe0 2023-06-23T18:04:03.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff3a00713f0 0x7ff3a0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00cc270 con 0x7ff3a00cd310 2023-06-23T18:04:03.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00cc550 con 0x7ff3a00713f0 2023-06-23T18:04:03.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a649f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00cc830 con 0x7ff3a01580e0 2023-06-23T18:04:03.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e759e700 1 --2- 172.21.15.22:0/3561443184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e80713f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009850 tx=0x7f26dc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631f038505c5c15d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a549d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966512632 0 0) 0x7fb2e8001110 con 0x7fb2ec0cde70 2023-06-23T18:04:03.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d80035d0 con 0x7fb2ec0cde70 2023-06-23T18:04:03.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a549d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60542/0 (socket says 172.21.15.22:60542) 2023-06-23T18:04:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a549d700 1 -- 172.21.15.22:0/1698798293 learned_addr learned my addr 172.21.15.22:0/1698798293 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 -- 172.21.15.22:0/4083702409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cd4e0 msgr2=0x7f6dd00cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 --2- 172.21.15.22:0/4083702409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cd4e0 0x7f6dd00cd940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00ffd0 tx=0x7f6dcc014850 comp rx=0 tx=0).stop 2023-06-23T18:04:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e5d9b700 1 -- 172.21.15.22:0/3561443184 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc011830 con 0x7f26e80cbbe0 2023-06-23T18:04:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e5d9b700 1 -- 172.21.15.22:0/3561443184 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc011990 con 0x7f26e80cbbe0 2023-06-23T18:04:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 -- 172.21.15.22:0/4083702409 shutdown_connections 2023-06-23T18:04:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 --2- 172.21.15.22:0/4083702409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cd4e0 0x7f6dd00cd940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 --2- 172.21.15.22:0/4083702409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd020 0x7f6dd00c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 -- 172.21.15.22:0/4083702409 >> 172.21.15.22:0/4083702409 conn(0x7f6dd006a5f0 msgr2=0x7f6dd00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 -- 172.21.15.22:0/2256606916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 msgr2=0x7fe31c0abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c0abe30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe310009910 tx=0x7fe310009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 -- 172.21.15.22:0/2382939755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c800afa90 msgr2=0x7f7c800aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c800afa90 0x7f7c800aff00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009df0 tx=0x7f7c7800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 -- 172.21.15.22:0/2382939755 shutdown_connections 2023-06-23T18:04:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c800afa90 0x7f7c800aff00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26e5d9b700 1 -- 172.21.15.22:0/3561443184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc011830 con 0x7f26e80cbbe0 2023-06-23T18:04:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/2382939755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c8006f890 0x7f7c8015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f7c7ffff700 1 -- 172.21.15.22:0/2382939755 >> 172.21.15.22:0/2382939755 conn(0x7f7c8006a5f0 msgr2=0x7f7c800caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 -- 172.21.15.22:0/4083702409 shutdown_connections 2023-06-23T18:04:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f6dd6174700 1 -- 172.21.15.22:0/4083702409 wait complete. 2023-06-23T18:04:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c2ffd700 1 -- 172.21.15.22:0/1466821133 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411369767 0 0) 0x7f10c40ce350 con 0x7f10c40cdde0 2023-06-23T18:04:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c2ffd700 1 -- 172.21.15.22:0/1466821133 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10ac0035d0 con 0x7f10c40cdde0 2023-06-23T18:04:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c3fff700 1 -- 172.21.15.22:0/1466821133 >> v1:172.21.15.169:6789/0 conn(0x7f10c40cf4b0 legacy=0x7f10c40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c3fff700 1 -- 172.21.15.22:0/1466821133 >> v1:172.21.15.22:6789/0 conn(0x7f10c40cdde0 legacy=0x7f10c40ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c3fff700 1 -- 172.21.15.22:0/1466821133 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40ce910 con 0x7f10c415fee0 2023-06-23T18:04:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c3fff700 1 --2- 172.21.15.22:0/1466821133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 0x7f10c40d0970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10b40098b0 tx=0x7f10b4009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8928c7e504164f6e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c2ffd700 1 -- 172.21.15.22:0/1466821133 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4004230 con 0x7f10c415fee0 2023-06-23T18:04:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c2ffd700 1 -- 172.21.15.22:0/1466821133 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b4004390 con 0x7f10c415fee0 2023-06-23T18:04:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c2ffd700 1 -- 172.21.15.22:0/1466821133 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40036c0 con 0x7f10c415fee0 2023-06-23T18:04:03.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a5c9e700 1 --2- 172.21.15.22:0/1698798293 >> v2:172.21.15.22:3300/0 conn(0x7ff3a00713f0 0x7ff3a0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183433202 0 0) 0x7ff3a00cc270 con 0x7ff3a00cd310 2023-06-23T18:04:03.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff38c0046e0 con 0x7ff3a00cd310 2023-06-23T18:04:03.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a5c9e700 1 -- 172.21.15.22:0/1698798293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 msgr2=0x7ff3a00cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26eca51700 1 -- 172.21.15.22:0/3561443184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 msgr2=0x7f26e80713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26eca51700 1 --2- 172.21.15.22:0/3561443184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e80713f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009850 tx=0x7f26dc009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:03.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4dffb700 1 -- 172.21.15.22:0/3709204597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48010070 con 0x7f2c500afa90 2023-06-23T18:04:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4dffb700 1 -- 172.21.15.22:0/3709204597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4800be90 con 0x7f2c500afa90 2023-06-23T18:04:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f1bcc700 1 --2- 172.21.15.22:0/418787941 >> v2:172.21.15.169:3300/0 conn(0x7fb2ec0cf1b0 0x7fb2ec0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379021870 0 0) 0x7fb2d80035d0 con 0x7fb2ec0cde70 2023-06-23T18:04:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 >> v2:172.21.15.169:3300/0 conn(0x7fb2ec0cf1b0 msgr2=0x7fb2ec0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 --2- 172.21.15.22:0/418787941 >> v2:172.21.15.169:3300/0 conn(0x7fb2ec0cf1b0 0x7fb2ec0cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 >> v2:172.21.15.22:3300/0 conn(0x7fb2ec0b5a70 msgr2=0x7fb2ec0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 --2- 172.21.15.22:0/418787941 >> v2:172.21.15.22:3300/0 conn(0x7fb2ec0b5a70 0x7fb2ec0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0cd3b0 con 0x7fb2ec0cde70 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fb2e8001800 con 0x7fb2ec0cde70 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 >> v1:172.21.15.169:6789/0 conn(0x7fb2ec0cde70 legacy=0x7fb2ec0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 0x7fb2d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004870 0x7fb2d8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2d8005220 0x7fb2d800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b62e0 con 0x7fb2d8005220 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cc830 con 0x7fb2d8004870 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d800c100 con 0x7fb2d8003a20 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 0x7fb2d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f1bcc700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004870 0x7fb2d8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fb2f2bce700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2d8005220 0x7fb2d800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 -- 172.21.15.22:0/2256606916 shutdown_connections 2023-06-23T18:04:03.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c0abe30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 -- 172.21.15.22:0/1466821133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 msgr2=0x7f10c40d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f10c9754700 1 --2- 172.21.15.22:0/1466821133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 0x7f10c40d0970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10b40098b0 tx=0x7f10b4009bc0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 --2- 172.21.15.22:0/2256606916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 -- 172.21.15.22:0/2256606916 >> 172.21.15.22:0/2256606916 conn(0x7fe31c069960 msgr2=0x7fe31c069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 -- 172.21.15.22:0/2256606916 shutdown_connections 2023-06-23T18:04:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7fe321686700 1 -- 172.21.15.22:0/2256606916 wait complete. 2023-06-23T18:04:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f2c4dffb700 1 -- 172.21.15.22:0/3709204597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800de30 con 0x7f2c500afa90 2023-06-23T18:04:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a5c9e700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3390697183 0 0) 0x7ff38c0046e0 con 0x7ff3a00cd310 2023-06-23T18:04:03.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a5c9e700 1 -- 172.21.15.22:0/1698798293 >> v1:172.21.15.169:6789/0 conn(0x7ff3a00cd310 legacy=0x7ff3a01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7ff3a5c9e700 1 -- 172.21.15.22:0/1698798293 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00ccb10 con 0x7ff3a00713f0 2023-06-23T18:04:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26eca51700 1 -- 172.21.15.22:0/3561443184 shutdown_connections 2023-06-23T18:04:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26eca51700 1 --2- 172.21.15.22:0/3561443184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e80713f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.267+0000 7f26eca51700 1 -- 172.21.15.22:0/3561443184 >> 172.21.15.22:0/3561443184 conn(0x7f26e806a5f0 msgr2=0x7f26e80caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a549d700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 -- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004870 msgr2=0x7fb2d8004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2d8004870 0x7fb2d8004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 -- 172.21.15.22:0/418787941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2d8005220 msgr2=0x7fb2d800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2d8005220 0x7fb2d800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 -- 172.21.15.22:0/418787941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0cd3b0 con 0x7fb2d8003a20 2023-06-23T18:04:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a5c9e700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 0x7ff3a0158bc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009850 tx=0x7ff39c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b046b20da18f2268 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980019f0 con 0x7ff3a00713f0 2023-06-23T18:04:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 Processor -- start 2023-06-23T18:04:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c004030 con 0x7ff3a00713f0 2023-06-23T18:04:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff397fff700 1 -- 172.21.15.22:0/1698798293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0043e0 con 0x7ff3a00713f0 2023-06-23T18:04:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 -- start start 2023-06-23T18:04:03.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- 172.21.15.22:0/2382939755 shutdown_connections 2023-06-23T18:04:03.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- 172.21.15.22:0/3561443184 shutdown_connections 2023-06-23T18:04:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- 172.21.15.22:0/3709204597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500afa90 msgr2=0x7f2c500afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/3709204597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500afa90 0x7f2c500afe90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c500cf110 tx=0x7f2c4800d650 comp rx=0 tx=0).stop 2023-06-23T18:04:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- 172.21.15.22:0/2382939755 wait complete. 2023-06-23T18:04:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- 172.21.15.22:0/3561443184 wait complete. 2023-06-23T18:04:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- 172.21.15.22:0/3709204597 shutdown_connections 2023-06-23T18:04:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/3709204597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500afa90 0x7f2c500afe90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/3709204597 >> v2:172.21.15.169:3300/0 conn(0x7f2c500cc880 0x7f2c5006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- 172.21.15.22:0/3709204597 >> 172.21.15.22:0/3709204597 conn(0x7f2c5006a5f0 msgr2=0x7f2c500caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 0x7fb2d8003e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c290 tx=0x7fb2d80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83858801596e5bf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f2bce700 1 -- 172.21.15.22:0/418787941 reap_dead start 2023-06-23T18:04:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8007d20 con 0x7fb2d8003a20 2023-06-23T18:04:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e800cd90 con 0x7fb2d8003a20 2023-06-23T18:04:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f13cb700 1 -- 172.21.15.22:0/418787941 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8007d20 con 0x7fb2d8003a20 2023-06-23T18:04:03.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f10c9754700 1 -- 172.21.15.22:0/1466821133 shutdown_connections 2023-06-23T18:04:03.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f10c9754700 1 --2- 172.21.15.22:0/1466821133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c415fee0 0x7f10c40d0970 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f10c9754700 1 -- 172.21.15.22:0/1466821133 >> 172.21.15.22:0/1466821133 conn(0x7f10c4069960 msgr2=0x7f10c4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 Processor -- start 2023-06-23T18:04:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 -- start start 2023-06-23T18:04:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 -- 172.21.15.22:0/1698798293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 msgr2=0x7ff3a0158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 0x7ff3a0158bc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009850 tx=0x7ff39c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- 172.21.15.22:0/3709204597 shutdown_connections 2023-06-23T18:04:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- 172.21.15.22:0/3709204597 wait complete. 2023-06-23T18:04:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 -- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 msgr2=0x7fb2d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 0x7fb2d8003e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c290 tx=0x7fb2d80058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 -- 172.21.15.22:0/1698798293 shutdown_connections 2023-06-23T18:04:03.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 0x7ff3a0158bc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 --2- 172.21.15.22:0/1698798293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a00cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 -- 172.21.15.22:0/1698798293 >> 172.21.15.22:0/1698798293 conn(0x7ff3a006a5f0 msgr2=0x7ff3a00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 -- 172.21.15.22:0/418787941 shutdown_connections 2023-06-23T18:04:03.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 --2- 172.21.15.22:0/418787941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2d8003a20 0x7fb2d8003e30 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c290 tx=0x7fb2d80058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 -- 172.21.15.22:0/418787941 >> 172.21.15.22:0/418787941 conn(0x7fb2ec069960 msgr2=0x7fb2ec069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 -- 172.21.15.22:0/1698798293 shutdown_connections 2023-06-23T18:04:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 Processor -- start 2023-06-23T18:04:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- start start 2023-06-23T18:04:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8006f890 0x7f7c80155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c80071660 0x7f7c801f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 0x7f7c80159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 Processor -- start 2023-06-23T18:04:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- start start 2023-06-23T18:04:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e8153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 0x7f26e80b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8158050 0x7f26e80c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c0c9d00 0x7fe31c0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0c3c80 con 0x7fe31c0ad380 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0c3000 con 0x7fe31c0c9d00 2023-06-23T18:04:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe321686700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0c32e0 con 0x7fe31c0cb130 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43954/0 (socket says 172.21.15.22:43954) 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 -- 172.21.15.22:0/2835772853 learned_addr learned my addr 172.21.15.22:0/2835772853 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31a7fc700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c0c9d00 0x7fe31c0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31b7fe700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 -- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 msgr2=0x7fe31c0abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe31c0cb130 0x7fe31c0abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 -- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c0c9d00 msgr2=0x7fe31c0c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c0c9d00 0x7fe31c0c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fe31affd700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009570 con 0x7fe31c0ad380 2023-06-23T18:04:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f10c9754700 1 -- 172.21.15.22:0/1466821133 shutdown_connections 2023-06-23T18:04:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f10c9754700 1 -- 172.21.15.22:0/1466821133 wait complete. 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 0x7f6dd01ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00cd020 0x7f6dd00b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd4e0 0x7f6dd00cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 0x7f6dd01ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01e3c90 con 0x7f6dd0071500 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01e33a0 con 0x7f6dd00cd020 2023-06-23T18:04:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd6174700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01e3640 con 0x7f6dd00cd4e0 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 0x7f6dd01ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43946/0 (socket says 172.21.15.22:43946) 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 -- 172.21.15.22:0/931609539 learned_addr learned my addr 172.21.15.22:0/931609539 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5973700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd4e0 0x7f6dd00cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 -- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd4e0 msgr2=0x7f6dd00cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00cd4e0 0x7f6dd00cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 -- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00cd020 msgr2=0x7f6dd00b3610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:04:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00cd020 0x7f6dd00b3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc00fdb0 con 0x7f6dd0071500 2023-06-23T18:04:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f6dd5172700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 0x7f6dd01ee070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800cbc0 tx=0x7f6dc800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801fe1e0 con 0x7f7c8006f890 2023-06-23T18:04:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80155f80 con 0x7f7c80071660 2023-06-23T18:04:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f7c7ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80201c40 con 0x7f7c800afa90 2023-06-23T18:04:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b2eb0 con 0x7f26e80cbbe0 2023-06-23T18:04:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b2230 con 0x7f26e8158050 2023-06-23T18:04:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f26eca51700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b2510 con 0x7f26e80cd310 2023-06-23T18:04:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 Processor -- start 2023-06-23T18:04:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7f2c4ffff700 1 -- start start 2023-06-23T18:04:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7ff3a649f700 1 -- 172.21.15.22:0/1698798293 wait complete. 2023-06-23T18:04:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.271+0000 7fb2f33cf700 1 -- 172.21.15.22:0/418787941 shutdown_connections 2023-06-23T18:04:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 -- 172.21.15.22:0/418787941 wait complete. 2023-06-23T18:04:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 0x7f7c80159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7e7fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c80071660 0x7f7c801f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 0x7f26e80b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 0x7f7c80159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60578/0 (socket says 172.21.15.22:60578) 2023-06-23T18:04:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/3800303092 learned_addr learned my addr 172.21.15.22:0/3800303092 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 0x7f26e80b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60594/0 (socket says 172.21.15.22:60594) 2023-06-23T18:04:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 -- 172.21.15.22:0/2896034834 learned_addr learned my addr 172.21.15.22:0/2896034834 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e759e700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8158050 0x7f26e80c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e6d9d700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e8153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 Processor -- start 2023-06-23T18:04:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 -- start start 2023-06-23T18:04:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 Processor -- start 2023-06-23T18:04:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/3800303092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c80071660 msgr2=0x7f7c801f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c80071660 0x7f7c801f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8006f890 msgr2=0x7f7c80155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8006f890 0x7f7c80155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c78009a70 con 0x7f7c800afa90 2023-06-23T18:04:03.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe31affd700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c150680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c500 tx=0x7fe30c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f7c7f7fe700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 0x7f7c80159990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800b370 tx=0x7f7c78004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0054e0 con 0x7fe31c0ad380 2023-06-23T18:04:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c0036a0 con 0x7fe31c0ad380 2023-06-23T18:04:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c003a50 con 0x7fe31c0ad380 2023-06-23T18:04:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c0c35c0 con 0x7fe31c0ad380 2023-06-23T18:04:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 Processor -- start 2023-06-23T18:04:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8003c20 con 0x7f6dd0071500 2023-06-23T18:04:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8003da0 con 0x7f6dd0071500 2023-06-23T18:04:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800ab40 con 0x7f6dd0071500 2023-06-23T18:04:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd01e3920 con 0x7f6dd0071500 2023-06-23T18:04:03.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd00cc7e0 con 0x7f6dd0071500 2023-06-23T18:04:03.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c500afa90 0x7f2c501fef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 0x7f2c50158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c500ccc50 0x7f2c501fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50158d70 con 0x7f2c500cc880 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50159820 con 0x7f2c500afa90 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500b1000 con 0x7f2c500ccc50 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 0x7f2c50158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 0x7f2c50158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43978/0 (socket says 172.21.15.22:43978) 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 -- 172.21.15.22:0/853754389 learned_addr learned my addr 172.21.15.22:0/853754389 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4f7fe700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c500ccc50 0x7f2c501fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 -- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c500ccc50 msgr2=0x7f2c501fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c500ccc50 0x7f2c501fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 -- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c500afa90 msgr2=0x7f2c501fef60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c500afa90 0x7f2c501fef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4800d050 con 0x7f2c500cc880 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4e7fc700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 0x7f2c50158830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c500 tx=0x7f2c4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440054e0 con 0x7f2c500cc880 2023-06-23T18:04:03.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c440036a0 con 0x7f2c500cc880 2023-06-23T18:04:03.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44003a50 con 0x7f2c500cc880 2023-06-23T18:04:03.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 -- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8158050 msgr2=0x7f26e80c1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8158050 0x7f26e80c1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 -- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 msgr2=0x7f26e8153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50158f80 con 0x7f2c500cc880 2023-06-23T18:04:03.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c50159540 con 0x7f2c500cc880 2023-06-23T18:04:03.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26e80cbbe0 0x7f26e8153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 -- 172.21.15.22:0/2896034834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0094b0 con 0x7f26e80cd310 2023-06-23T18:04:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26e659c700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 0x7f26e80b1850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c530 tx=0x7f26d800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 0x7ff3a01e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3a00cd310 0x7ff3a01e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80052d0 con 0x7f26e80cd310 2023-06-23T18:04:03.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b1ab0 con 0x7ff3a00713f0 2023-06-23T18:04:03.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b0ee0 con 0x7ff3a00cd310 2023-06-23T18:04:03.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a649f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b1180 con 0x7ff3a01580e0 2023-06-23T18:04:03.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a5c9e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a5c9e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60602/0 (socket says 172.21.15.22:60602) 2023-06-23T18:04:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a5c9e700 1 -- 172.21.15.22:0/613139625 learned_addr learned my addr 172.21.15.22:0/613139625 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8005430 con 0x7f26e80cd310 2023-06-23T18:04:03.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80055a0 con 0x7f26e80cd310 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 -- start start 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c40cdde0 0x7f10c415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c40cf4b0 0x7f10c4162930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10c415fee0 0x7f10c4162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c415e1d0 con 0x7f10c415fee0 2023-06-23T18:04:03.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c415d320 con 0x7f10c40cdde0 2023-06-23T18:04:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c9754700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40c96a0 con 0x7f10c40cf4b0 2023-06-23T18:04:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c150bc0 con 0x7fe31c0ad380 2023-06-23T18:04:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c40cdde0 0x7f10c415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c40cdde0 0x7f10c415cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:42882/0 (socket says 172.21.15.22:42882) 2023-06-23T18:04:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c3fff700 1 -- 172.21.15.22:0/482054621 learned_addr learned my addr 172.21.15.22:0/482054621 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f10c37fe700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c40cf4b0 0x7f10c4162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 -- start start 2023-06-23T18:04:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec1f54c0 0x7fb2ec15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 0x7fb2ec0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0b4f50 0x7fb2ec0afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15a220 con 0x7fb2ec15ba50 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15b080 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7fb2f33cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15a470 con 0x7fb2ec0b4f50 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3a00cd310 0x7ff3a01e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 -- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 msgr2=0x7ff3a006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3a01580e0 0x7ff3a006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 -- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 msgr2=0x7ff3a01e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a00713f0 0x7ff3a01e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7ff3a4c9c700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0094b0 con 0x7ff3a00cd310 2023-06-23T18:04:03.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f1bcc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 0x7fb2ec0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f1bcc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 0x7fb2ec0b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44000/0 (socket says 172.21.15.22:44000) 2023-06-23T18:04:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f1bcc700 1 -- 172.21.15.22:0/1148144294 learned_addr learned my addr 172.21.15.22:0/1148144294 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe31c0cef30 con 0x7fe31c0ad380 2023-06-23T18:04:03.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78025610 con 0x7f7c800afa90 2023-06-23T18:04:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78011040 con 0x7f7c800afa90 2023-06-23T18:04:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c80201f20 con 0x7f7c800afa90 2023-06-23T18:04:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff3a4c9c700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3a00cd310 0x7ff3a01e0b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff398001700 tx=0x7ff39800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c802024e0 con 0x7f7c800afa90 2023-06-23T18:04:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec1f54c0 0x7fb2ec15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c8f53700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10c415fee0 0x7f10c4162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398003e30 con 0x7ff3a00cd310 2023-06-23T18:04:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398005070 con 0x7ff3a00cd310 2023-06-23T18:04:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00b1460 con 0x7ff3a00cd310 2023-06-23T18:04:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a01e3180 con 0x7ff3a00cd310 2023-06-23T18:04:03.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780259b0 con 0x7f7c800afa90 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 -- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c40cdde0 msgr2=0x7f10c415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c40cdde0 0x7f10c415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 -- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10c415fee0 msgr2=0x7f10c4162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10c415fee0 0x7f10c4162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b40094b0 con 0x7f10c40cf4b0 2023-06-23T18:04:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c8f53700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10c415fee0 0x7f10c4162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:03.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c800ca240 con 0x7f7c800afa90 2023-06-23T18:04:03.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe30c003bb0 con 0x7fe31c0ad380 2023-06-23T18:04:03.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e80b27f0 con 0x7f26e80cd310 2023-06-23T18:04:03.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.275+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e8153e70 con 0x7f26e80cd310 2023-06-23T18:04:03.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6dd00c8dc0 con 0x7f6dd0071500 2023-06-23T18:04:03.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6dc800aca0 con 0x7f6dd0071500 2023-06-23T18:04:03.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6db40662f0 0x7f6db40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6dc8084020 con 0x7f6dd0071500 2023-06-23T18:04:03.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] conn(0x7f6db406fec0 0x7f6db40722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6db40729c0 con 0x7f6db406fec0 2023-06-23T18:04:03.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe3037fe700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3080662f0 0x7fe3080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dd4971700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6db40662f0 0x7f6db40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc80511b0 con 0x7f6dd0071500 2023-06-23T18:04:03.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c50071360 con 0x7f2c500cc880 2023-06-23T18:04:03.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398005420 con 0x7ff3a00cd310 2023-06-23T18:04:03.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff380000fc0 con 0x7ff3a00cd310 2023-06-23T18:04:03.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f26d80038c0 con 0x7f26e80cd310 2023-06-23T18:04:03.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 -- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0b4f50 msgr2=0x7fb2ec0afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0b4f50 0x7fb2ec0afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 -- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 msgr2=0x7fb2ec0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 0x7fb2ec0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec15a750 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f1bcc700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec15ba50 0x7fb2ec0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f23cd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec1f54c0 0x7fb2ec15b510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8016000 tx=0x7fb2e8000fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8005320 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8005480 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8006aa0 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec15aa30 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1f5890 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2c44003bb0 con 0x7f2c500cc880 2023-06-23T18:04:03.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dd4971700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6db40662f0 0x7f6db40687a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0157e70 tx=0x7f6dc0006c30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dd5973700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] conn(0x7f6db406fec0 0x7f6db40722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f10c37fe700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c40cf4b0 0x7f10c4162930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c500 tx=0x7f10b800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe31a7fc700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3080662f0 0x7fe3080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff398005580 con 0x7ff3a00cd310 2023-06-23T18:04:03.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26cf7fe700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26d40662f0 0x7f26d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26d8083310 con 0x7f26e80cd310 2023-06-23T18:04:03.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f2c37fff700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c3c0663c0 0x7f2c3c068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff384066320 0x7ff3840687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26eca51700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] conn(0x7f26e80b1d90 0x7f26e80c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 --> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26e80cc270 con 0x7f26e80b1d90 2023-06-23T18:04:03.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe30c057c10 con 0x7fe31c0ad380 2023-06-23T18:04:03.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f6dd5973700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] conn(0x7f6db406fec0 0x7f6db40722f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff3980837e0 con 0x7ff3a00cd310 2023-06-23T18:04:03.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f26e6d9d700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26d40662f0 0x7f26d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe3037fe700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] conn(0x7fe30806fec0 0x7fe3080722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3080729c0 con 0x7fe30806fec0 2023-06-23T18:04:03.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c0011e0 con 0x7fe31c0ad380 2023-06-23T18:04:03.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2c44083540 con 0x7f2c500cc880 2023-06-23T18:04:03.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.279+0000 7f2c37fff700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] conn(0x7f2c3c070090 0x7f2c3c0724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff3a549d700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff384066320 0x7ff3840687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff395ffb700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] conn(0x7ff38406fff0 0x7ff384072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff384072af0 con 0x7ff38406fff0 2023-06-23T18:04:03.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff398085e00 con 0x7ff3a00cd310 2023-06-23T18:04:03.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fe31b7fe700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] conn(0x7fe30806fec0 0x7fe3080722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7c78025b10 con 0x7f7c800afa90 2023-06-23T18:04:03.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff3a5c9e700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] conn(0x7ff38406fff0 0x7ff384072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8005d90 con 0x7f10c40cf4b0 2023-06-23T18:04:03.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b800ce70 con 0x7f10c40cf4b0 2023-06-23T18:04:03.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c9754700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c415d550 con 0x7f10c40cf4b0 2023-06-23T18:04:03.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b80072d0 con 0x7f10c40cf4b0 2023-06-23T18:04:03.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c9754700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c415db10 con 0x7f10c40cf4b0 2023-06-23T18:04:03.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7c6c066340 0x7f7c6c0687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7c780861f0 con 0x7f7c800afa90 2023-06-23T18:04:03.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f26e6d9d700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26d40662f0 0x7f26d40687a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26e80ace00 tx=0x7f26e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff3a549d700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff384066320 0x7ff3840687d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff3a5c9e700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] conn(0x7ff38406fff0 0x7ff384072420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] conn(0x7f7c6c070010 0x7f7c6c072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c6c072b10 con 0x7f7c6c070010 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78086560 con 0x7f7c800afa90 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c7effd700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7c6c066340 0x7f7c6c0687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c4f7fe700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] conn(0x7f2c3c070090 0x7f2c3c0724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c4effd700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c3c0663c0 0x7f2c3c068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c072b90 con 0x7f2c3c070090 2023-06-23T18:04:03.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c44085aa0 con 0x7f2c500cc880 2023-06-23T18:04:03.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== osd.7 v2:172.21.15.169:6800/802398196 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6db40729c0 con 0x7f6db406fec0 2023-06-23T18:04:03.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c7effd700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7c6c066340 0x7f7c6c0687f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c8006ff10 tx=0x7f7c74007af0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c7e7fc700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] conn(0x7f7c6c070010 0x7f7c6c072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fe31a7fc700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3080662f0 0x7fe3080687a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0c1cf0 tx=0x7fe3140098d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fe31b7fe700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] conn(0x7fe30806fec0 0x7fe3080722f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c9754700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10a4000fc0 con 0x7f10c40cf4b0 2023-06-23T18:04:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c4f7fe700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] conn(0x7f2c3c070090 0x7f2c3c0724c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f2c4effd700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c3c0663c0 0x7f2c3c068870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c400099d0 tx=0x7f2c400091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2ec0ccd60 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb2e8022020 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== osd.0 v2:172.21.15.22:6808/445321366 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff384072af0 con 0x7ff38406fff0 2023-06-23T18:04:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== osd.3 v2:172.21.15.22:6824/1664577612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe3080729c0 con 0x7fe30806fec0 2023-06-23T18:04:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d0066320 0x7fb2d00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb2e8084c40 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2f1bcc700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d0066320 0x7fb2d00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] conn(0x7fb2d006ff70 0x7fb2d00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d0072a70 con 0x7fb2d006ff70 2023-06-23T18:04:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e8087300 con 0x7fb2ec1f54c0 2023-06-23T18:04:03.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f26e759e700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] conn(0x7f26e80b1d90 0x7f26e80c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2f2bce700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] conn(0x7fb2d006ff70 0x7fb2d00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f10b8007430 con 0x7f10c40cf4b0 2023-06-23T18:04:03.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2f1bcc700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d0066320 0x7fb2d00687d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b57d0 tx=0x7fb2dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7fb2f2bce700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] conn(0x7fb2d006ff70 0x7fb2d00723a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f10c0ff9700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f10b00662f0 0x7f10b00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c7e7fc700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] conn(0x7f7c6c070010 0x7f7c6c072440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.283+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== osd.1 v2:172.21.15.22:6809/3293388284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7c6c072b10 con 0x7f7c6c070010 2023-06-23T18:04:03.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== osd.5 v2:172.21.15.169:6816/1436778973 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2c3c072b90 con 0x7f2c3c070090 2023-06-23T18:04:03.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c3fff700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f10b00662f0 0x7f10b00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f10b8083380 con 0x7f10c40cf4b0 2023-06-23T18:04:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c0ff9700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] conn(0x7f10b006ffc0 0x7f10b00723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10b0072ac0 con 0x7f10b006ffc0 2023-06-23T18:04:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b80859a0 con 0x7f10c40cf4b0 2023-06-23T18:04:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f26e759e700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] conn(0x7f26e80b1d90 0x7f26e80c22b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== osd.6 v2:172.21.15.169:6824/3045974108 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb2d0072a70 con 0x7fb2d006ff70 2023-06-23T18:04:03.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== osd.2 v2:172.21.15.22:6800/3001036907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f26dc022040 con 0x7f26e80b1d90 2023-06-23T18:04:03.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c8f53700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] conn(0x7f10b006ffc0 0x7f10b00723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c3fff700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f10b00662f0 0x7f10b00687a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097d0 tx=0x7f10b4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c8f53700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] conn(0x7f10b006ffc0 0x7f10b00723f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.287+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== osd.4 v2:172.21.15.169:6806/1771639099 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f10b0072ac0 con 0x7f10b006ffc0 2023-06-23T18:04:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.303+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 --> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c500cc2f0 con 0x7f2c3c070090 2023-06-23T18:04:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.303+0000 7f2c37fff700 1 -- 172.21.15.22:0/853754389 <== osd.5 v2:172.21.15.169:6816/1436778973 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c48007c40 con 0x7f2c3c070090 2023-06-23T18:04:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.307+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 --> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26e80cc830 con 0x7f26e80b1d90 2023-06-23T18:04:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.307+0000 7f26cf7fe700 1 -- 172.21.15.22:0/2896034834 <== osd.2 v2:172.21.15.22:6800/3001036907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26dc006c80 con 0x7f26e80b1d90 2023-06-23T18:04:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] conn(0x7f2c3c070090 msgr2=0x7f2c3c0724c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.169:6816/1436778973,v1:172.21.15.169:6817/1436778973] conn(0x7f2c3c070090 0x7f2c3c0724c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c3c0663c0 msgr2=0x7f2c3c068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c3c0663c0 0x7f2c3c068870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c400099d0 tx=0x7f2c400091c0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 --> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6dd00c8ae0 con 0x7f6db406fec0 2023-06-23T18:04:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 msgr2=0x7f2c50158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 --2- 172.21.15.22:0/853754389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c500cc880 0x7f2c50158830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c500 tx=0x7f2c4400c8c0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] conn(0x7f26e80b1d90 msgr2=0x7f26e80c22b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6800/3001036907,v1:172.21.15.22:6801/3001036907] conn(0x7f26e80b1d90 0x7f26e80c22b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4f7fe700 1 -- 172.21.15.22:0/853754389 reap_dead start 2023-06-23T18:04:03.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 shutdown_connections 2023-06-23T18:04:03.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 >> 172.21.15.22:0/853754389 conn(0x7f2c5006a5f0 msgr2=0x7f2c500ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 shutdown_connections 2023-06-23T18:04:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26d40662f0 msgr2=0x7f26d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26d40662f0 0x7f26d40687a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26e80ace00 tx=0x7f26e0009040 comp rx=0 tx=0).stop 2023-06-23T18:04:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 msgr2=0x7f26e80b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f26eca51700 1 --2- 172.21.15.22:0/2896034834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26e80cd310 0x7f26e80b1850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c530 tx=0x7f26d800c840 comp rx=0 tx=0).stop 2023-06-23T18:04:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 --> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe31c0ce730 con 0x7fe30806fec0 2023-06-23T18:04:03.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f2c4ffff700 1 -- 172.21.15.22:0/853754389 wait complete. 2023-06-23T18:04:03.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 --> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c800ca800 con 0x7f7c6c070010 2023-06-23T18:04:03.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.311+0000 7f6dc5ffb700 1 -- 172.21.15.22:0/931609539 <== osd.7 v2:172.21.15.169:6800/802398196 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6dcc00fc30 con 0x7f6db406fec0 2023-06-23T18:04:03.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe3037fe700 1 -- 172.21.15.22:0/2835772853 <== osd.3 v2:172.21.15.22:6824/1664577612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe310004500 con 0x7fe30806fec0 2023-06-23T18:04:03.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] conn(0x7f6db406fec0 msgr2=0x7f6db40722f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.169:6800/802398196,v1:172.21.15.169:6801/802398196] conn(0x7f6db406fec0 0x7f6db40722f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6db40662f0 msgr2=0x7f6db40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6db40662f0 0x7f6db40687a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0157e70 tx=0x7f6dc0006c30 comp rx=0 tx=0).stop 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 msgr2=0x7f6dd01ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 --2- 172.21.15.22:0/931609539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd0071500 0x7f6dd01ee070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800cbc0 tx=0x7f6dc800ced0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f26e759e700 1 -- 172.21.15.22:0/2896034834 reap_dead start 2023-06-23T18:04:03.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd5973700 1 -- 172.21.15.22:0/931609539 reap_dead start 2023-06-23T18:04:03.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c677fe700 1 -- 172.21.15.22:0/3800303092 <== osd.1 v2:172.21.15.22:6809/3293388284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c7000da10 con 0x7f7c6c070010 2023-06-23T18:04:03.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 shutdown_connections 2023-06-23T18:04:03.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 >> 172.21.15.22:0/2896034834 conn(0x7f26e806a5f0 msgr2=0x7f26e8154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 shutdown_connections 2023-06-23T18:04:03.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 >> 172.21.15.22:0/931609539 conn(0x7f6dd006a5f0 msgr2=0x7f6dd00c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 shutdown_connections 2023-06-23T18:04:03.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f6dd6174700 1 -- 172.21.15.22:0/931609539 wait complete. 2023-06-23T18:04:03.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 shutdown_connections 2023-06-23T18:04:03.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] conn(0x7f7c6c070010 msgr2=0x7f7c6c072440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6809/3293388284,v1:172.21.15.22:6811/3293388284] conn(0x7f7c6c070010 0x7f7c6c072440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f26eca51700 1 -- 172.21.15.22:0/2896034834 wait complete. 2023-06-23T18:04:03.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7c6c066340 msgr2=0x7f7c6c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7c6c066340 0x7f7c6c0687f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c8006ff10 tx=0x7f7c74007af0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 msgr2=0x7f7c80159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 --2- 172.21.15.22:0/3800303092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c800afa90 0x7f7c80159990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800b370 tx=0x7f7c78004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7f7fe700 1 -- 172.21.15.22:0/3800303092 reap_dead start 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 shutdown_connections 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 >> 172.21.15.22:0/3800303092 conn(0x7f7c8006a5f0 msgr2=0x7f7c801566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 shutdown_connections 2023-06-23T18:04:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7f7c7ffff700 1 -- 172.21.15.22:0/3800303092 wait complete. 2023-06-23T18:04:03.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] conn(0x7fe30806fec0 msgr2=0x7fe3080722f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6824/1664577612,v1:172.21.15.22:6825/1664577612] conn(0x7fe30806fec0 0x7fe3080722f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3080662f0 msgr2=0x7fe3080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3080662f0 0x7fe3080687a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0c1cf0 tx=0x7fe3140098d0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 msgr2=0x7fe31c150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 --2- 172.21.15.22:0/2835772853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c0ad380 0x7fe31c150680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c500 tx=0x7fe30c00c810 comp rx=0 tx=0).stop 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe31b7fe700 1 -- 172.21.15.22:0/2835772853 reap_dead start 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 shutdown_connections 2023-06-23T18:04:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 >> 172.21.15.22:0/2835772853 conn(0x7fe31c069960 msgr2=0x7fe31c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 shutdown_connections 2023-06-23T18:04:03.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.315+0000 7fe321686700 1 -- 172.21.15.22:0/2835772853 wait complete. 2023-06-23T18:04:03.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 --> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2ec0cca80 con 0x7fb2d006ff70 2023-06-23T18:04:03.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2e2ffd700 1 -- 172.21.15.22:0/1148144294 <== osd.6 v2:172.21.15.169:6824/3045974108 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2e400f040 con 0x7fb2d006ff70 2023-06-23T18:04:03.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] conn(0x7fb2d006ff70 msgr2=0x7fb2d00723a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:6824/3045974108,v1:172.21.15.169:6825/3045974108] conn(0x7fb2d006ff70 0x7fb2d00723a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d0066320 msgr2=0x7fb2d00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d0066320 0x7fb2d00687d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b57d0 tx=0x7fb2dc008040 comp rx=0 tx=0).stop 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec1f54c0 msgr2=0x7fb2ec15b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 --2- 172.21.15.22:0/1148144294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec1f54c0 0x7fb2ec15b510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8016000 tx=0x7fb2e8000fe0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f2bce700 1 -- 172.21.15.22:0/1148144294 reap_dead start 2023-06-23T18:04:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 shutdown_connections 2023-06-23T18:04:03.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.319+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 >> 172.21.15.22:0/1148144294 conn(0x7fb2ec069960 msgr2=0x7fb2ec0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.327+0000 7f10c9754700 1 -- 172.21.15.22:0/482054621 --> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10a4002d60 con 0x7f10b006ffc0 2023-06-23T18:04:03.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.327+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 shutdown_connections 2023-06-23T18:04:03.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.327+0000 7fb2f33cf700 1 -- 172.21.15.22:0/1148144294 wait complete. 2023-06-23T18:04:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.327+0000 7f10c0ff9700 1 -- 172.21.15.22:0/482054621 <== osd.4 v2:172.21.15.169:6806/1771639099 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f10a4002d60 con 0x7f10b006ffc0 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] conn(0x7f10b006ffc0 msgr2=0x7f10b00723f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:6806/1771639099,v1:172.21.15.169:6808/1771639099] conn(0x7f10b006ffc0 0x7f10b00723f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f10b00662f0 msgr2=0x7f10b00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f10b00662f0 0x7f10b00687a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097d0 tx=0x7f10b4008030 comp rx=0 tx=0).stop 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c40cf4b0 msgr2=0x7f10c4162930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 --2- 172.21.15.22:0/482054621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c40cf4b0 0x7f10c4162930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c500 tx=0x7f10b800c810 comp rx=0 tx=0).stop 2023-06-23T18:04:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10c8f53700 1 -- 172.21.15.22:0/482054621 reap_dead start 2023-06-23T18:04:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 shutdown_connections 2023-06-23T18:04:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 >> 172.21.15.22:0/482054621 conn(0x7f10c4069960 msgr2=0x7f10c40cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 shutdown_connections 2023-06-23T18:04:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.331+0000 7f10aa7fc700 1 -- 172.21.15.22:0/482054621 wait complete. 2023-06-23T18:04:03.442 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.442 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2023-06-23T18:04:03.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-23T18:04:03.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-23T18:04:03.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.335+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 --> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff380002d60 con 0x7ff38406fff0 2023-06-23T18:04:03.445 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-23T18:04:03.446 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-23T18:04:03.447 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-23T18:04:03.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff395ffb700 1 -- 172.21.15.22:0/613139625 <== osd.0 v2:172.21.15.22:6808/445321366 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff380002d60 con 0x7ff38406fff0 2023-06-23T18:04:03.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] conn(0x7ff38406fff0 msgr2=0x7ff384072420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6808/445321366,v1:172.21.15.22:6810/445321366] conn(0x7ff38406fff0 0x7ff384072420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff384066320 msgr2=0x7ff3840687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff384066320 0x7ff3840687d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390008040 comp rx=0 tx=0).stop 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3a00cd310 msgr2=0x7ff3a01e0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 --2- 172.21.15.22:0/613139625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3a00cd310 0x7ff3a01e0b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff398001700 tx=0x7ff39800cb70 comp rx=0 tx=0).stop 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a5c9e700 1 -- 172.21.15.22:0/613139625 reap_dead start 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 shutdown_connections 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 >> 172.21.15.22:0/613139625 conn(0x7ff3a006a5f0 msgr2=0x7ff3a00ada20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 shutdown_connections 2023-06-23T18:04:03.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.339+0000 7ff3a649f700 1 -- 172.21.15.22:0/613139625 wait complete. 2023-06-23T18:04:03.451 INFO:teuthology.orchestra.run.smithi022.stdout:55834574858 2023-06-23T18:04:03.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-23T18:04:03.452 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-23T18:04:03.454 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2023-06-23T18:04:03.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-23T18:04:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.635+0000 7fc3ceb7b700 1 Processor -- start 2023-06-23T18:04:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.635+0000 7fc3ceb7b700 1 -- start start 2023-06-23T18:04:03.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.635+0000 7ffa09960700 1 Processor -- start 2023-06-23T18:04:03.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.635+0000 7ffa09960700 1 -- start start 2023-06-23T18:04:03.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ceb7b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 0x7fc3c80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ceb7b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 0x7fc3c80affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ceb7b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c80cde50 con 0x7fc3c8070db0 2023-06-23T18:04:03.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ceb7b700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c815ae90 con 0x7fc3c806f660 2023-06-23T18:04:03.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ceb7b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c815a020 con 0x7fc3c80cda80 2023-06-23T18:04:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cd378700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 0x7fc3c80affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cd378700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 0x7fc3c80affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:42892/0 (socket says 172.21.15.22:42892) 2023-06-23T18:04:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cd378700 1 -- 172.21.15.22:0/1335800354 learned_addr learned my addr 172.21.15.22:0/1335800354 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 Processor -- start 2023-06-23T18:04:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 Processor -- start 2023-06-23T18:04:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 -- start start 2023-06-23T18:04:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fd76ffff700 1 Processor -- start 2023-06-23T18:04:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 -- start start 2023-06-23T18:04:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fd76ffff700 1 -- start start 2023-06-23T18:04:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa04071500 0x7ffa040c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa040cd4e0 0x7ffa040cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa040cde80 con 0x7ffa04071500 2023-06-23T18:04:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0415c270 con 0x7ffa040cd4e0 2023-06-23T18:04:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa09960700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040b02e0 con 0x7ffa040cd020 2023-06-23T18:04:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa04071500 0x7ffa040c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffa04071500 0x7ffa040c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44008/0 (socket says 172.21.15.22:44008) 2023-06-23T18:04:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 -- 172.21.15.22:0/3112446145 learned_addr learned my addr 172.21.15.22:0/3112446145 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa03fff700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0915f700 1 --2- 172.21.15.22:0/3112446145 >> v2:172.21.15.169:3300/0 conn(0x7ffa040cd4e0 0x7ffa040cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 -- 172.21.15.22:0/3112446145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 msgr2=0x7ffa040afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 -- 172.21.15.22:0/3112446145 >> v2:172.21.15.169:3300/0 conn(0x7ffa040cd4e0 msgr2=0x7ffa040cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 --2- 172.21.15.22:0/3112446145 >> v2:172.21.15.169:3300/0 conn(0x7ffa040cd4e0 0x7ffa040cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 -- 172.21.15.22:0/3112446145 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040c8800 con 0x7ffa04071500 2023-06-23T18:04:03.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa03fff700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:03.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f67a80cb1a0 0x7f67a806d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f67a80cc120 0x7f67a80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a81582c0 con 0x7f67a80cc120 2023-06-23T18:04:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a806d540 con 0x7f67a80cb1a0 2023-06-23T18:04:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 Processor -- start 2023-06-23T18:04:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5f0071500 0x7fa5f00c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f015c270 con 0x7fa5f00cd020 2023-06-23T18:04:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f00b02e0 con 0x7fa5f00cd4e0 2023-06-23T18:04:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5effff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00c8800 con 0x7fa5f0071500 2023-06-23T18:04:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/1335800354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 0x7fc3c80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ccb77700 1 -- 172.21.15.22:0/1335800354 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243228124 0 0) 0x7fc3c80cde50 con 0x7fc3c8070db0 2023-06-23T18:04:03.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ccb77700 1 -- 172.21.15.22:0/1335800354 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b00035d0 con 0x7fc3c8070db0 2023-06-23T18:04:03.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67acb13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8071920 con 0x7f67a80cc880 2023-06-23T18:04:03.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 -- start start 2023-06-23T18:04:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 -- 172.21.15.22:0/1335800354 >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 msgr2=0x7fc3c80affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/1335800354 >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 0x7fc3c80affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 -- 172.21.15.22:0/1335800354 >> v1:172.21.15.22:6789/0 conn(0x7fc3c8070db0 legacy=0x7fc3c8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 -- 172.21.15.22:0/1335800354 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c815a210 con 0x7fc3c80cda80 2023-06-23T18:04:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5f0071500 0x7fa5f00c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5f0071500 0x7fa5f00c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:42916/0 (socket says 172.21.15.22:42916) 2023-06-23T18:04:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2368417540 learned_addr learned my addr 172.21.15.22:0/2368417540 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60634/0 (socket says 172.21.15.22:60634) 2023-06-23T18:04:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a77fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f67a80cb1a0 0x7f67a806d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 -- 172.21.15.22:0/2883056655 learned_addr learned my addr 172.21.15.22:0/2883056655 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/1335800354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 0x7fc3c80afa90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009840 tx=0x7fc3bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee61c750f177d89 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa0895e700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040c92e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa040cbb70 tx=0x7ff9fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d5fef4a556e22d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a7fff700 1 --2- 172.21.15.22:0/2883056655 >> v2:172.21.15.22:3300/0 conn(0x7f67a80cc120 0x7f67a80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2368417540 >> v1:172.21.15.169:6789/0 conn(0x7fa5f00cd4e0 legacy=0x7fa5f00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2368417540 >> v1:172.21.15.22:6789/0 conn(0x7fa5f00cd020 legacy=0x7fa5f00afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2368417540 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00c8ae0 con 0x7fa5f0071500 2023-06-23T18:04:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 -- 172.21.15.22:0/2883056655 >> v2:172.21.15.169:3300/0 conn(0x7f67a80cb1a0 msgr2=0x7f67a806d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 --2- 172.21.15.22:0/2883056655 >> v2:172.21.15.169:3300/0 conn(0x7f67a80cb1a0 0x7f67a806d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 -- 172.21.15.22:0/2883056655 >> v2:172.21.15.22:3300/0 conn(0x7f67a80cc120 msgr2=0x7f67a80afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 --2- 172.21.15.22:0/2883056655 >> v2:172.21.15.22:3300/0 conn(0x7f67a80cc120 0x7f67a80afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 -- 172.21.15.22:0/2883056655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8071360 con 0x7f67a80cc880 2023-06-23T18:04:03.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa037fe700 1 -- 172.21.15.22:0/3112446145 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00e070 con 0x7ffa04071500 2023-06-23T18:04:03.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ccb77700 1 -- 172.21.15.22:0/1335800354 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc005e50 con 0x7fc3c80cda80 2023-06-23T18:04:03.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ccb77700 1 -- 172.21.15.22:0/1335800354 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc006030 con 0x7fc3c80cda80 2023-06-23T18:04:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0c0cda80 0x7feb0c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fc3ccb77700 1 -- 172.21.15.22:0/1335800354 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0063a0 con 0x7fc3c80cda80 2023-06-23T18:04:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fa5eeffd700 1 --2- 172.21.15.22:0/2368417540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f0071500 0x7fa5f00c92e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009910 tx=0x7fa5e4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc3408a0477a612 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f0d07a8a700 1 Processor -- start 2023-06-23T18:04:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7fe096ab5700 1 Processor -- start 2023-06-23T18:04:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f0d07a8a700 1 -- start start 2023-06-23T18:04:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe096ab5700 1 -- start start 2023-06-23T18:04:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5edffb700 1 -- 172.21.15.22:0/2368417540 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005e50 con 0x7fa5f0071500 2023-06-23T18:04:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5edffb700 1 -- 172.21.15.22:0/2368417540 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4009e60 con 0x7fa5f0071500 2023-06-23T18:04:03.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5edffb700 1 -- 172.21.15.22:0/2368417540 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4004250 con 0x7fa5f0071500 2023-06-23T18:04:03.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb0c06f660 0x7feb0c0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0cde50 con 0x7feb0c070db0 2023-06-23T18:04:03.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c15ae90 con 0x7feb0c0cda80 2023-06-23T18:04:03.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb11fe3700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c15a020 con 0x7feb0c06f660 2023-06-23T18:04:03.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb10fe1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0c0cda80 0x7feb0c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb10fe1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0c0cda80 0x7feb0c0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44026/0 (socket says 172.21.15.22:44026) 2023-06-23T18:04:03.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7feb10fe1700 1 -- 172.21.15.22:0/3258076445 learned_addr learned my addr 172.21.15.22:0/3258076445 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb0bfff700 1 --2- 172.21.15.22:0/3258076445 >> v2:172.21.15.169:3300/0 conn(0x7feb0c06f660 0x7feb0c0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb10fe1700 1 -- 172.21.15.22:0/3258076445 >> v2:172.21.15.169:3300/0 conn(0x7feb0c06f660 msgr2=0x7feb0c0affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb10fe1700 1 --2- 172.21.15.22:0/3258076445 >> v2:172.21.15.169:3300/0 conn(0x7feb0c06f660 0x7feb0c0affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa037fe700 1 -- 172.21.15.22:0/3112446145 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0090d0 con 0x7ffa04071500 2023-06-23T18:04:03.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7ffa037fe700 1 -- 172.21.15.22:0/3112446145 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00bb50 con 0x7ffa04071500 2023-06-23T18:04:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb10fe1700 1 -- 172.21.15.22:0/3258076445 >> v1:172.21.15.169:6789/0 conn(0x7feb0c070db0 legacy=0x7feb0c071230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:04:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb10fe1700 1 -- 172.21.15.22:0/3258076445 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c15a210 con 0x7feb0c0cda80 2023-06-23T18:04:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- 172.21.15.22:0/3112446145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 msgr2=0x7ffa040c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040c92e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa040cbb70 tx=0x7ff9fc00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb10fe1700 1 --2- 172.21.15.22:0/3258076445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c0cda80 0x7feb0c0afa90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cf110 tx=0x7feafc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49695bbca2674be3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/1335800354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 msgr2=0x7fc3c80afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/1335800354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 0x7fc3c80afa90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009840 tx=0x7fc3bc009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.639+0000 7f67a6ffd700 1 --2- 172.21.15.22:0/2883056655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67a80cf110 tx=0x7f679c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b98301fdaa0b465 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67a67fc700 1 -- 172.21.15.22:0/2883056655 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c011830 con 0x7f67a80cc880 2023-06-23T18:04:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd770164920 0x7fd7701613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- 172.21.15.22:0/3112446145 shutdown_connections 2023-06-23T18:04:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 --2- 172.21.15.22:0/3112446145 >> v2:172.21.15.169:3300/0 conn(0x7ffa040cd4e0 0x7ffa040cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 --2- 172.21.15.22:0/3112446145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040c92e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- 172.21.15.22:0/3112446145 >> 172.21.15.22:0/3112446145 conn(0x7ffa0406a5f0 msgr2=0x7ffa040caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd7701601f0 0x7fd770070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7700ce990 con 0x7fd77015fe20 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd770160f20 con 0x7fd7701601f0 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd770160940 con 0x7fd770164920 2023-06-23T18:04:03.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76effd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd77015fe20 0x7fd7700ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37176/0 (socket says 172.21.15.22:37176) 2023-06-23T18:04:03.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76effd700 1 -- 172.21.15.22:0/3862103464 learned_addr learned my addr 172.21.15.22:0/3862103464 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:03.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/3862103464 >> v2:172.21.15.169:3300/0 conn(0x7fd770164920 0x7fd7701613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/1335800354 shutdown_connections 2023-06-23T18:04:03.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/1335800354 >> v2:172.21.15.169:3300/0 conn(0x7fc3c806f660 0x7fc3c80affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/1335800354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80cda80 0x7fc3c80afa90 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009840 tx=0x7fc3bc009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76dffb700 1 -- 172.21.15.22:0/3862103464 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322740117 0 0) 0x7fd758001110 con 0x7fd77015fe20 2023-06-23T18:04:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76f7fe700 1 --2- 172.21.15.22:0/3862103464 >> v2:172.21.15.22:3300/0 conn(0x7fd7701601f0 0x7fd770070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/1335800354 >> 172.21.15.22:0/1335800354 conn(0x7fc3c806a5f0 msgr2=0x7fc3c80caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76dffb700 1 -- 172.21.15.22:0/3862103464 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd75c0035d0 con 0x7fd77015fe20 2023-06-23T18:04:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 -- 172.21.15.22:0/3862103464 >> v2:172.21.15.22:3300/0 conn(0x7fd7701601f0 msgr2=0x7fd770070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- 172.21.15.22:0/3112446145 shutdown_connections 2023-06-23T18:04:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5effff700 1 -- 172.21.15.22:0/2368417540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f0071500 msgr2=0x7fa5f00c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5effff700 1 --2- 172.21.15.22:0/2368417540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f0071500 0x7fa5f00c92e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009910 tx=0x7fa5e4009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb0b7fe700 1 -- 172.21.15.22:0/3258076445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc005e50 con 0x7feb0c0cda80 2023-06-23T18:04:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67a67fc700 1 -- 172.21.15.22:0/2883056655 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c011990 con 0x7f67a80cc880 2023-06-23T18:04:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb0b7fe700 1 -- 172.21.15.22:0/3258076445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc004030 con 0x7feb0c0cda80 2023-06-23T18:04:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/1335800354 shutdown_connections 2023-06-23T18:04:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb0b7fe700 1 -- 172.21.15.22:0/3258076445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0043a0 con 0x7feb0c0cda80 2023-06-23T18:04:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67a67fc700 1 -- 172.21.15.22:0/2883056655 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c011d20 con 0x7f67a80cc880 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d07a8a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d000ce0c0 0x7f0d000ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d07a8a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0006b8b0 con 0x7f0d0015d840 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d07a8a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000c9630 con 0x7f0d000b3a90 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d07a8a700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000c9fc0 con 0x7f0d000ce0c0 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d000ce0c0 0x7f0d000ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d000ce0c0 0x7f0d000ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44044/0 (socket says 172.21.15.22:44044) 2023-06-23T18:04:03.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 -- 172.21.15.22:0/2264266207 learned_addr learned my addr 172.21.15.22:0/2264266207 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe096ab5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0900ccc50 0x7fe0900cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe096ab5700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe090071920 con 0x7fe0900cc120 2023-06-23T18:04:03.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe096ab5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe090071360 con 0x7fe0900cc880 2023-06-23T18:04:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe096ab5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0901582c0 con 0x7fe0900ccc50 2023-06-23T18:04:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/1335800354 wait complete. 2023-06-23T18:04:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0900ccc50 0x7fe0900cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0900ccc50 0x7fe0900cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44050/0 (socket says 172.21.15.22:44050) 2023-06-23T18:04:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 -- 172.21.15.22:0/856687215 learned_addr learned my addr 172.21.15.22:0/856687215 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 -- 172.21.15.22:0/2883056655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 msgr2=0x7f67a80cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 -- 172.21.15.22:0/856687215 >> v1:172.21.15.169:6789/0 conn(0x7fe0900cc880 legacy=0x7fe09006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 --2- 172.21.15.22:0/2883056655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67a80cf110 tx=0x7f679c009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 -- 172.21.15.22:0/2264266207 >> v1:172.21.15.169:6789/0 conn(0x7f0d000b3a90 legacy=0x7f0d000c8f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:04:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 -- 172.21.15.22:0/2264266207 >> v1:172.21.15.22:6789/0 conn(0x7f0d0015d840 legacy=0x7f0d000c8810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:04:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 -- 172.21.15.22:0/2264266207 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000ce9d0 con 0x7f0d000ce0c0 2023-06-23T18:04:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d05826700 1 --2- 172.21.15.22:0/2264266207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d000ce0c0 0x7f0d000ce490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009e50 tx=0x7f0cf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d8a2ac01dfc000 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 -- 172.21.15.22:0/3258076445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c0cda80 msgr2=0x7feb0c0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3258076445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c0cda80 0x7feb0c0afa90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cf110 tx=0x7feafc009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 -- 172.21.15.22:0/3258076445 shutdown_connections 2023-06-23T18:04:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3258076445 >> v2:172.21.15.169:3300/0 conn(0x7feb0c06f660 0x7feb0c0affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3258076445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c0cda80 0x7feb0c0afa90 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cf110 tx=0x7feafc009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 -- 172.21.15.22:0/3258076445 >> 172.21.15.22:0/3258076445 conn(0x7feb0c06a5f0 msgr2=0x7feb0c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- 172.21.15.22:0/3112446145 wait complete. 2023-06-23T18:04:03.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 Processor -- start 2023-06-23T18:04:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 -- 172.21.15.22:0/856687215 >> v1:172.21.15.22:6789/0 conn(0x7fe0900cc120 legacy=0x7fe0900c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 -- 172.21.15.22:0/856687215 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090071640 con 0x7fe0900ccc50 2023-06-23T18:04:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 Processor -- start 2023-06-23T18:04:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fc3ceb7b700 1 -- start start 2023-06-23T18:04:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 -- 172.21.15.22:0/2883056655 shutdown_connections 2023-06-23T18:04:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 --2- 172.21.15.22:0/2883056655 >> v2:172.21.15.22:3300/0 conn(0x7f67a80cc120 0x7f67a80afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 --2- 172.21.15.22:0/2883056655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc880 0x7f67a80cbbe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 --2- 172.21.15.22:0/2883056655 >> v2:172.21.15.169:3300/0 conn(0x7f67a80cb1a0 0x7f67a806d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5effff700 1 -- 172.21.15.22:0/2368417540 shutdown_connections 2023-06-23T18:04:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5effff700 1 --2- 172.21.15.22:0/2368417540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f0071500 0x7fa5f00c92e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fa5effff700 1 -- 172.21.15.22:0/2368417540 >> 172.21.15.22:0/2368417540 conn(0x7fa5f006a5f0 msgr2=0x7fa5f00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 -- 172.21.15.22:0/2883056655 >> 172.21.15.22:0/2883056655 conn(0x7f67a806a5f0 msgr2=0x7f67a80caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe0952b2700 1 --2- 172.21.15.22:0/856687215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900ccc50 0x7fe0900cbbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe088009e50 tx=0x7fe08800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e95b50ff32a6cb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 -- 172.21.15.22:0/2883056655 shutdown_connections 2023-06-23T18:04:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f67acb13700 1 -- 172.21.15.22:0/2883056655 wait complete. 2023-06-23T18:04:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 -- 172.21.15.22:0/3258076445 shutdown_connections 2023-06-23T18:04:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe094ab1700 1 -- 172.21.15.22:0/856687215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800e070 con 0x7fe0900ccc50 2023-06-23T18:04:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe094ab1700 1 -- 172.21.15.22:0/856687215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08800b9c0 con 0x7fe0900ccc50 2023-06-23T18:04:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fe094ab1700 1 -- 172.21.15.22:0/856687215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800bd50 con 0x7fe0900ccc50 2023-06-23T18:04:03.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7feb11fe3700 1 -- 172.21.15.22:0/3258076445 wait complete. 2023-06-23T18:04:03.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d04824700 1 -- 172.21.15.22:0/2264266207 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800e070 con 0x7f0d000ce0c0 2023-06-23T18:04:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d04824700 1 -- 172.21.15.22:0/2264266207 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8003a40 con 0x7f0d000ce0c0 2023-06-23T18:04:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7f0d04824700 1 -- 172.21.15.22:0/2264266207 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8003df0 con 0x7f0d000ce0c0 2023-06-23T18:04:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 0x7fc3c815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 -- 172.21.15.22:0/856687215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900ccc50 msgr2=0x7fe0900cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 --2- 172.21.15.22:0/856687215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900ccc50 0x7fe0900cbbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe088009e50 tx=0x7fe08800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7ffa09960700 1 -- start start 2023-06-23T18:04:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa040cd4e0 0x7ffa040c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040b0080 con 0x7ffa04071500 2023-06-23T18:04:03.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040c7e10 con 0x7ffa040cd4e0 2023-06-23T18:04:03.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040c5550 con 0x7ffa040cd020 2023-06-23T18:04:03.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 0x7fc3c80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/3862103464 >> v2:172.21.15.22:3300/0 conn(0x7fd7701601f0 0x7fd770070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 -- 172.21.15.22:0/3862103464 >> v1:172.21.15.22:6789/0 conn(0x7fd77015fe20 legacy=0x7fd7700ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 -- 172.21.15.22:0/856687215 shutdown_connections 2023-06-23T18:04:03.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 --2- 172.21.15.22:0/856687215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900ccc50 0x7fe0900cbbe0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe088009e50 tx=0x7fe08800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:03.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 -- 172.21.15.22:0/856687215 >> 172.21.15.22:0/856687215 conn(0x7fe09006a5f0 msgr2=0x7fe0900caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44068/0 (socket says 172.21.15.22:44068) 2023-06-23T18:04:03.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 -- 172.21.15.22:0/2368417540 shutdown_connections 2023-06-23T18:04:03.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 -- 172.21.15.22:0/209410027 learned_addr learned my addr 172.21.15.22:0/209410027 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 -- 172.21.15.22:0/2368417540 wait complete. 2023-06-23T18:04:03.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80b2600 con 0x7fc3c806f660 2023-06-23T18:04:03.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa03fff700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80b1820 con 0x7fc3c8070db0 2023-06-23T18:04:03.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ceb7b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80b1b00 con 0x7fc3c80b1380 2023-06-23T18:04:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 -- 172.21.15.22:0/3862103464 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770160670 con 0x7fd770164920 2023-06-23T18:04:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/3862103464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd7701613b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd770070540 tx=0x7fd768009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131c4a5d003cb37 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76dffb700 1 -- 172.21.15.22:0/3862103464 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768011830 con 0x7fd770164920 2023-06-23T18:04:03.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76dffb700 1 -- 172.21.15.22:0/3862103464 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd768011990 con 0x7fd770164920 2023-06-23T18:04:03.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- 172.21.15.22:0/3862103464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 msgr2=0x7fd7701613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 --2- 172.21.15.22:0/3862103464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd7701613b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd770070540 tx=0x7fd768009b30 comp rx=0 tx=0).stop 2023-06-23T18:04:03.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- 172.21.15.22:0/3862103464 shutdown_connections 2023-06-23T18:04:03.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 --2- 172.21.15.22:0/3862103464 >> v2:172.21.15.22:3300/0 conn(0x7fd7701601f0 0x7fd770070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 --2- 172.21.15.22:0/3862103464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd7701613b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- 172.21.15.22:0/3862103464 >> 172.21.15.22:0/3862103464 conn(0x7fd77006aa70 msgr2=0x7fd77006ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- 172.21.15.22:0/3862103464 shutdown_connections 2023-06-23T18:04:03.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.643+0000 7fd76ffff700 1 -- 172.21.15.22:0/3862103464 wait complete. 2023-06-23T18:04:03.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 -- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 msgr2=0x7ffa040c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 -- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa040cd4e0 msgr2=0x7ffa040c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa040cd4e0 0x7ffa040c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 -- 172.21.15.22:0/209410027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80094b0 con 0x7ffa04071500 2023-06-23T18:04:03.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa0895e700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa040b0810 tx=0x7ff9fc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 Processor -- start 2023-06-23T18:04:03.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 -- start start 2023-06-23T18:04:03.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f67acb13700 1 Processor -- start 2023-06-23T18:04:03.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 -- 172.21.15.22:0/2264266207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d000ce0c0 msgr2=0x7f0d000ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/2264266207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d000ce0c0 0x7f0d000ce490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009e50 tx=0x7f0cf800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:03.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 -- 172.21.15.22:0/2264266207 shutdown_connections 2023-06-23T18:04:03.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/2264266207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d000ce0c0 0x7f0d000ce490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 -- 172.21.15.22:0/2264266207 >> 172.21.15.22:0/2264266207 conn(0x7f0d00069960 msgr2=0x7f0d00069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 -- 172.21.15.22:0/2264266207 shutdown_connections 2023-06-23T18:04:03.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7f0d07a8a700 1 -- 172.21.15.22:0/2264266207 wait complete. 2023-06-23T18:04:03.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fd76ffff700 1 Processor -- start 2023-06-23T18:04:03.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0099b0 con 0x7ffa04071500 2023-06-23T18:04:03.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc004040 con 0x7ffa04071500 2023-06-23T18:04:03.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ce37a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ce37a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:60658/0 (socket says 172.21.15.22:60658) 2023-06-23T18:04:03.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ce37a700 1 -- 172.21.15.22:0/2778704107 learned_addr learned my addr 172.21.15.22:0/2778704107 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 0x7fc3c815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 -- 172.21.15.22:0/209410027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc009b30 con 0x7ffa04071500 2023-06-23T18:04:03.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fc3ce37a700 1 -- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 msgr2=0x7fc3c80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0043b0 con 0x7ffa04071500 2023-06-23T18:04:03.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 Processor -- start 2023-06-23T18:04:03.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7ffa09960700 1 -- 172.21.15.22:0/209410027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa040c5b10 con 0x7ffa04071500 2023-06-23T18:04:03.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 -- start start 2023-06-23T18:04:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb0c06f660 0x7feb0c098ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb0c1f5020 0x7feb0c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca180 con 0x7feb0c070db0 2023-06-23T18:04:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f54c0 con 0x7feb0c1f5020 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb11fe3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c9600 con 0x7feb0c06f660 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fd76ffff700 1 -- start start 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77015fe20 0x7fd7701585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd770149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7701566e0 con 0x7fd7701601f0 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd770149690 con 0x7fd770164920 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd770156f40 con 0x7fd77015fe20 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44092/0 (socket says 172.21.15.22:44092) 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 -- 172.21.15.22:0/2268470669 learned_addr learned my addr 172.21.15.22:0/2268470669 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76f7fe700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd770149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76effd700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77015fe20 0x7fd7701585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 -- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77015fe20 msgr2=0x7fd7701585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77015fe20 0x7fd7701585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 -- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 msgr2=0x7fd770149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd770149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 -- 172.21.15.22:0/2268470669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680094b0 con 0x7fd7701601f0 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76e7fc700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd768004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76801adc0 con 0x7fd7701601f0 2023-06-23T18:04:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 -- 172.21.15.22:0/856687215 shutdown_connections 2023-06-23T18:04:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fe096ab5700 1 -- 172.21.15.22:0/856687215 wait complete. 2023-06-23T18:04:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 Processor -- start 2023-06-23T18:04:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa09960700 1 -- 172.21.15.22:0/209410027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0404e4c0 con 0x7ffa04071500 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff9fc004510 con 0x7ffa04071500 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa017fa700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 0x7ff9f00687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff9fc084a20 con 0x7ffa04071500 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa03fff700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 0x7ff9f00687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 Processor -- start 2023-06-23T18:04:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 -- start start 2023-06-23T18:04:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 -- start start 2023-06-23T18:04:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44072/0 (socket says 172.21.15.22:44072) 2023-06-23T18:04:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb0bfff700 1 -- 172.21.15.22:0/3636672267 learned_addr learned my addr 172.21.15.22:0/3636672267 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb117e2700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb0c1f5020 0x7feb0c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7feb10fe1700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb0c06f660 0x7feb0c098ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 -- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb0c06f660 msgr2=0x7feb0c098ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb0c06f660 0x7feb0c098ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 -- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb0c1f5020 msgr2=0x7feb0c0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb0c1f5020 0x7feb0c0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 -- 172.21.15.22:0/3636672267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009520 con 0x7feb0c070db0 2023-06-23T18:04:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7feb0bfff700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feb0000ea00 tx=0x7feb0000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f00cd020 0x7fa5f0154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.647+0000 7fa5effff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f00cd4e0 0x7fa5f0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5effff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0156b30 con 0x7fa5f0071500 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5effff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0156e10 con 0x7fa5f00cd4e0 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5effff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01570f0 con 0x7fa5f00cd020 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44088/0 (socket says 172.21.15.22:44088) 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2297782086 learned_addr learned my addr 172.21.15.22:0/2297782086 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5ef7fe700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f00cd4e0 0x7fa5f0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f00cd020 msgr2=0x7fa5f0154120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-23T18:04:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f00cd020 0x7fa5f0154120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f00cd4e0 msgr2=0x7fa5f0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f00cd4e0 0x7fa5f0157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 -- 172.21.15.22:0/2297782086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e4009520 con 0x7fa5f0071500 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fa5eeffd700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009fd0 tx=0x7fa5e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3cd378700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 0x7fc3c80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ce37a700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 0x7fc3c80b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ce37a700 1 -- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 msgr2=0x7fc3c815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ce37a700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 0x7fc3c815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ce37a700 1 -- 172.21.15.22:0/2778704107 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc009450 con 0x7fc3c80b1380 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 0x7fc3c815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3cd378700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 0x7fc3c80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ce37a700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c400 tx=0x7fc3b800c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7ffa03fff700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 0x7ff9f00687f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa040ac8c0 tx=0x7ff9f8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8015d70 con 0x7fc3c80b1380 2023-06-23T18:04:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b80050d0 con 0x7fc3c80b1380 2023-06-23T18:04:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80b1de0 con 0x7fc3c80b1380 2023-06-23T18:04:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8005480 con 0x7fc3c80b1380 2023-06-23T18:04:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c8159a60 con 0x7fc3c80b1380 2023-06-23T18:04:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7680243f0 con 0x7fd7701601f0 2023-06-23T18:04:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680247a0 con 0x7fd7701601f0 2023-06-23T18:04:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 -- 172.21.15.22:0/2268470669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77015b740 con 0x7fd7701601f0 2023-06-23T18:04:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fd76ffff700 1 -- 172.21.15.22:0/2268470669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd770156bd0 con 0x7fd7701601f0 2023-06-23T18:04:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc120 0x7f67a80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 0x7f67a814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80ac9b0 con 0x7f67a80cc880 2023-06-23T18:04:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a814e290 con 0x7f67a80cb1a0 2023-06-23T18:04:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67acb13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80b10d0 con 0x7f67a80cc120 2023-06-23T18:04:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 -- start start 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0900cc880 0x7fe0900c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0900c3bb0 0x7fe0901ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901ee480 con 0x7fe0900cc120 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901ee760 con 0x7fe0900c3bb0 2023-06-23T18:04:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe096ab5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900c2b90 con 0x7fe0900cc880 2023-06-23T18:04:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67a77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67a7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 0x7f67a814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67a77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:43018/0 (socket says 172.21.15.22:43018) 2023-06-23T18:04:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f67a77fe700 1 -- 172.21.15.22:0/2997051575 learned_addr learned my addr 172.21.15.22:0/2997051575 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe095ab3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe095ab3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44134/0 (socket says 172.21.15.22:44134) 2023-06-23T18:04:03.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7fe095ab3700 1 -- 172.21.15.22:0/2861337486 learned_addr learned my addr 172.21.15.22:0/2861337486 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d000b3a90 0x7f0d000c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d000ce0c0 0x7f0d000c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000c4890 con 0x7f0d0015d840 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00158dd0 con 0x7f0d000b3a90 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d07a8a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001490d0 con 0x7f0d000ce0c0 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44108/0 (socket says 172.21.15.22:44108) 2023-06-23T18:04:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe0962b4700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0900c3bb0 0x7fe0901ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 -- 172.21.15.22:0/3406603885 learned_addr learned my addr 172.21.15.22:0/3406603885 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 -- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d000ce0c0 msgr2=0x7f0d000c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d000ce0c0 0x7f0d000c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 -- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d000b3a90 msgr2=0x7f0d000c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d000b3a90 0x7f0d000c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 -- 172.21.15.22:0/3406603885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009b30 con 0x7f0d0015d840 2023-06-23T18:04:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.651+0000 7f0d06027700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b410 tx=0x7f0cfc00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 -- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc120 msgr2=0x7f67a80ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc120 0x7f67a80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 -- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 msgr2=0x7f67a80ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 -- 172.21.15.22:0/2997051575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009570 con 0x7f67a80cc880 2023-06-23T18:04:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c8159c20 con 0x7fc3c80b1380 2023-06-23T18:04:03.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 -- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0900cc880 msgr2=0x7fe0900c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0900cc880 0x7fe0900c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 -- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0900c3bb0 msgr2=0x7fe0901ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0900c3bb0 0x7fe0901ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 -- 172.21.15.22:0/2861337486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088009b30 con 0x7fe0900cc120 2023-06-23T18:04:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a77fe700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67a7fff700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 0x7f67a814d6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67a000eb70 tx=0x7f67a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe095ab3700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c460 tx=0x7fe08000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc3b80036a0 con 0x7fc3c80b1380 2023-06-23T18:04:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3c67fc700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 0x7fc3b40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc3b808bcc0 con 0x7fc3c80b1380 2023-06-23T18:04:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 0x7fc3b40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fc3cdb79700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 0x7fc3b40687a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80ca9b0 tx=0x7fc3bc008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00009720 con 0x7feb0c070db0 2023-06-23T18:04:03.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4003d40 con 0x7fa5f0071500 2023-06-23T18:04:03.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0c98e0 con 0x7feb0c070db0 2023-06-23T18:04:03.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f01573d0 con 0x7fa5f0071500 2023-06-23T18:04:03.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f0154f90 con 0x7fa5f0071500 2023-06-23T18:04:03.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c0c9ea0 con 0x7feb0c070db0 2023-06-23T18:04:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff9fc051d20 con 0x7ffa04071500 2023-06-23T18:04:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800052d0 con 0x7fe0900cc120 2023-06-23T18:04:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000ca00 con 0x7f67a80cc880 2023-06-23T18:04:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd76ffff700 1 -- 172.21.15.22:0/2268470669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd754005310 con 0x7fd7701601f0 2023-06-23T18:04:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd768011b00 con 0x7fd7701601f0 2023-06-23T18:04:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67acb13700 1 -- 172.21.15.22:0/2997051575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a80b3900 con 0x7f67a80cc880 2023-06-23T18:04:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7f67acb13700 1 -- 172.21.15.22:0/2997051575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a814e4a0 con 0x7f67a80cc880 2023-06-23T18:04:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000eea0 con 0x7feb0c070db0 2023-06-23T18:04:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00005c40 con 0x7feb0c070db0 2023-06-23T18:04:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400b610 con 0x7fa5f0071500 2023-06-23T18:04:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.655+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4003ea0 con 0x7fa5f0071500 2023-06-23T18:04:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec005310 con 0x7feb0c070db0 2023-06-23T18:04:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe080005430 con 0x7fe0900cc120 2023-06-23T18:04:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800055a0 con 0x7fe0900cc120 2023-06-23T18:04:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d0005310 con 0x7fa5f0071500 2023-06-23T18:04:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0900c2e70 con 0x7fe0900cc120 2023-06-23T18:04:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09014dbf0 con 0x7fe0900cc120 2023-06-23T18:04:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd767fff700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 0x7fd750068820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd76effd700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 0x7fd750068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd768085600 con 0x7fd7701601f0 2023-06-23T18:04:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7fd76effd700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 0x7fd750068820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd758001270 tx=0x7fd758009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feb0000c850 con 0x7feb0c070db0 2023-06-23T18:04:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb097fa700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 0x7feaf80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb000843b0 con 0x7feb0c070db0 2023-06-23T18:04:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb10fe1700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 0x7feaf80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.659+0000 7feb10fe1700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 0x7feaf80687a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb0c071b10 tx=0x7feafc008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000f040 con 0x7f67a80cc880 2023-06-23T18:04:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000cd60 con 0x7f67a80cc880 2023-06-23T18:04:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc022640 con 0x7f0d0015d840 2023-06-23T18:04:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6788005310 con 0x7f67a80cc880 2023-06-23T18:04:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0227a0 con 0x7f0d0015d840 2023-06-23T18:04:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc022b10 con 0x7f0d0015d840 2023-06-23T18:04:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d001493b0 con 0x7f0d0015d840 2023-06-23T18:04:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d00149970 con 0x7f0d0015d840 2023-06-23T18:04:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa5e4018030 con 0x7fa5f0071500 2023-06-23T18:04:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fa5d77fe700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 0x7fa5dc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5e4085830 con 0x7fa5f0071500 2023-06-23T18:04:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fa5ee7fc700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 0x7fa5dc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0900c3390 con 0x7fe0900cc120 2023-06-23T18:04:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc3b8058fc0 con 0x7fc3c80b1380 2023-06-23T18:04:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fa5ee7fc700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 0x7fa5dc0687a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00c7610 tx=0x7fa5e8001710 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.663+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd768052a20 con 0x7fd7701601f0 2023-06-23T18:04:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe0800038c0 con 0x7fe0900cc120 2023-06-23T18:04:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7fe0867fc700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 0x7fe0740687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7fe0952b2700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 0x7fe0740687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7fe0952b2700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 0x7fe0740687a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0900cd060 tx=0x7fe088010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7feb00054fd0 con 0x7feb0c070db0 2023-06-23T18:04:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.667+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00149b90 con 0x7f0d0015d840 2023-06-23T18:04:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.671+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa5e4052c50 con 0x7fa5f0071500 2023-06-23T18:04:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.675+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0cfc022c70 con 0x7f0d0015d840 2023-06-23T18:04:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.675+0000 7f0cf67fc700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 0x7f0ce40687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.675+0000 7f0d05826700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 0x7f0ce40687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.675+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0cfc095290 con 0x7f0d0015d840 2023-06-23T18:04:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.675+0000 7f0d05826700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 0x7f0ce40687f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b370 tx=0x7f0cf8010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.679+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f67a0014480 con 0x7f67a80cc880 2023-06-23T18:04:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.679+0000 7f678ffff700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 0x7f67940687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.679+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f67a0084f50 con 0x7f67a80cc880 2023-06-23T18:04:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.679+0000 7f67a77fe700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 0x7f67940687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.679+0000 7f67a77fe700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 0x7f67940687a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67980112c0 tx=0x7f67980091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.683+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe080058570 con 0x7fe0900cc120 2023-06-23T18:04:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.683+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe0800508f0 con 0x7fe0900cc120 2023-06-23T18:04:03.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.683+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0cfc062590 con 0x7f0d0015d840 2023-06-23T18:04:03.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.695+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f67a0052040 con 0x7f67a80cc880 2023-06-23T18:04:03.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.971+0000 7fd76ffff700 1 -- 172.21.15.22:0/2268470669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd754005890 con 0x7fd7701601f0 2023-06-23T18:04:03.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.975+0000 7fd767fff700 1 -- 172.21.15.22:0/2268470669 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd768056040 con 0x7fd7701601f0 2023-06-23T18:04:03.991 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 msgr2=0x7fd750068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 0x7fd750068820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd758001270 tx=0x7fd758009040 comp rx=0 tx=0).stop 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 msgr2=0x7fd77015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd768004770 comp rx=0 tx=0).stop 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 shutdown_connections 2023-06-23T18:04:03.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd750066370 0x7fd750068820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd770164920 0x7fd770149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7701601f0 0x7fd77015b0f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 --2- 172.21.15.22:0/2268470669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77015fe20 0x7fd7701585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 >> 172.21.15.22:0/2268470669 conn(0x7fd77006aa70 msgr2=0x7fd7700b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 shutdown_connections 2023-06-23T18:04:03.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7fd765ffb700 1 -- 172.21.15.22:0/2268470669 wait complete. 2023-06-23T18:04:03.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7ffa09960700 1 -- 172.21.15.22:0/209410027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa040c9b30 con 0x7ffa04071500 2023-06-23T18:04:03.996 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:03.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.979+0000 7ffa017fa700 1 -- 172.21.15.22:0/209410027 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff9fc055340 con 0x7ffa04071500 2023-06-23T18:04:04.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 msgr2=0x7ff9f00687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 0x7ff9f00687f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa040ac8c0 tx=0x7ff9f8009240 comp rx=0 tx=0).stop 2023-06-23T18:04:04.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 msgr2=0x7ffa040afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa040b0810 tx=0x7ff9fc004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 shutdown_connections 2023-06-23T18:04:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff9f0066340 0x7ff9f00687f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa040cd4e0 0x7ffa040c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa040cd020 0x7ffa040c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 --2- 172.21.15.22:0/209410027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa04071500 0x7ffa040afb40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 >> 172.21.15.22:0/209410027 conn(0x7ffa0406a5f0 msgr2=0x7ffa040cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 shutdown_connections 2023-06-23T18:04:04.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.987+0000 7ff9eaffd700 1 -- 172.21.15.22:0/209410027 wait complete. 2023-06-23T18:04:04.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc3c80b2a90 con 0x7fc3c80b1380 2023-06-23T18:04:04.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-06-23T18:04:04.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3c67fc700 1 -- 172.21.15.22:0/2778704107 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3c80b2a90 con 0x7fc3c80b1380 2023-06-23T18:04:04.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa5d0005890 con 0x7fa5f0071500 2023-06-23T18:04:04.012 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.013 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2023-06-23T18:04:04.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feaec005180 con 0x7feb0c070db0 2023-06-23T18:04:04.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fa5d77fe700 1 -- 172.21.15.22:0/2297782086 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5e4056270 con 0x7fa5f0071500 2023-06-23T18:04:04.014 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:04.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7feb097fa700 1 -- 172.21.15.22:0/3636672267 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb00054b60 con 0x7feb0c070db0 2023-06-23T18:04:04.014 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:04.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 msgr2=0x7fc3b40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 0x7fc3b40687a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80ca9b0 tx=0x7fc3bc008070 comp rx=0 tx=0).stop 2023-06-23T18:04:04.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 msgr2=0x7fc3c8098a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.995+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c400 tx=0x7fc3b800c7c0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 shutdown_connections 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c80b1380 0x7fc3c8098a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc3b40662f0 0x7fc3b40687a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c8070db0 0x7fc3c80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 --2- 172.21.15.22:0/2778704107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3c806f660 0x7fc3c815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 >> 172.21.15.22:0/2778704107 conn(0x7fc3c806a5f0 msgr2=0x7fc3c815dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 shutdown_connections 2023-06-23T18:04:04.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fc3ceb7b700 1 -- 172.21.15.22:0/2778704107 wait complete. 2023-06-23T18:04:04.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 msgr2=0x7feaf80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 0x7feaf80687a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb0c071b10 tx=0x7feafc008070 comp rx=0 tx=0).stop 2023-06-23T18:04:04.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 msgr2=0x7feb0c1f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feb0000ea00 tx=0x7feb0000ed10 comp rx=0 tx=0).stop 2023-06-23T18:04:04.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0d0004dc70 con 0x7f0d0015d840 2023-06-23T18:04:04.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-06-23T18:04:04.021 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 shutdown_connections 2023-06-23T18:04:04.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feaf80662f0 0x7feaf80687a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb0c1f5020 0x7feb0c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0c070db0 0x7feb0c1f4ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 --2- 172.21.15.22:0/3636672267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb0c06f660 0x7feb0c098ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 >> 172.21.15.22:0/3636672267 conn(0x7feb0c06a5f0 msgr2=0x7feb0c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 shutdown_connections 2023-06-23T18:04:04.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7feb11fe3700 1 -- 172.21.15.22:0/3636672267 wait complete. 2023-06-23T18:04:04.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0cf67fc700 1 -- 172.21.15.22:0/3406603885 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0cfc065bb0 con 0x7f0d0015d840 2023-06-23T18:04:04.025 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2023-06-23T18:04:04.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 msgr2=0x7fa5dc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 0x7fa5dc0687a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00c7610 tx=0x7fa5e8001710 comp rx=0 tx=0).stop 2023-06-23T18:04:04.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 msgr2=0x7fa5f0154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009fd0 tx=0x7fa5e4004770 comp rx=0 tx=0).stop 2023-06-23T18:04:04.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 shutdown_connections 2023-06-23T18:04:04.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa5dc0662f0 0x7fa5dc0687a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f00cd4e0 0x7fa5f0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f00cd020 0x7fa5f0154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 --2- 172.21.15.22:0/2297782086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0071500 0x7fa5f0154a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 >> 172.21.15.22:0/2297782086 conn(0x7fa5f006a5f0 msgr2=0x7fa5f006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 shutdown_connections 2023-06-23T18:04:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7fa5effff700 1 -- 172.21.15.22:0/2297782086 wait complete. 2023-06-23T18:04:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 msgr2=0x7f0ce40687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 0x7f0ce40687f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b370 tx=0x7f0cf8010000 comp rx=0 tx=0).stop 2023-06-23T18:04:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 msgr2=0x7f0d001f9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b410 tx=0x7f0cfc00b7d0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 shutdown_connections 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0015d840 0x7f0d001f9660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d000ce0c0 0x7f0d000c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ce4066340 0x7f0ce40687f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 --2- 172.21.15.22:0/3406603885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d000b3a90 0x7f0d000c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 >> 172.21.15.22:0/3406603885 conn(0x7f0d00069960 msgr2=0x7f0d001593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 shutdown_connections 2023-06-23T18:04:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:03.999+0000 7f0d07a8a700 1 -- 172.21.15.22:0/3406603885 wait complete. 2023-06-23T18:04:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6788005890 con 0x7f67a80cc880 2023-06-23T18:04:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678ffff700 1 -- 172.21.15.22:0/2997051575 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67a0018090 con 0x7f67a80cc880 2023-06-23T18:04:04.032 INFO:teuthology.orchestra.run.smithi022.stdout:55834574858 2023-06-23T18:04:04.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 msgr2=0x7f67940687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 0x7f67940687a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67980112c0 tx=0x7f67980091c0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 msgr2=0x7f67a814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.007+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 0x7f67a814d6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67a000eb70 tx=0x7f67a000ee80 comp rx=0 tx=0).stop 2023-06-23T18:04:04.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 shutdown_connections 2023-06-23T18:04:04.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a80cc880 0x7f67a814d6f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67a80cc120 0x7f67a80ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f67940662f0 0x7f67940687a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 --2- 172.21.15.22:0/2997051575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67a80cb1a0 0x7f67a80ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 >> 172.21.15.22:0/2997051575 conn(0x7f67a806a5f0 msgr2=0x7f67a806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 shutdown_connections 2023-06-23T18:04:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7f678dffb700 1 -- 172.21.15.22:0/2997051575 wait complete. 2023-06-23T18:04:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe0900c3390 con 0x7fe0900cc120 2023-06-23T18:04:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.011+0000 7fe0867fc700 1 -- 172.21.15.22:0/2861337486 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe080017020 con 0x7fe0900cc120 2023-06-23T18:04:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2023-06-23T18:04:04.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-06-23T18:04:04.038 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 msgr2=0x7fe0740687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 0x7fe0740687a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0900cd060 tx=0x7fe088010000 comp rx=0 tx=0).stop 2023-06-23T18:04:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 msgr2=0x7fe09014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c460 tx=0x7fe08000c770 comp rx=0 tx=0).stop 2023-06-23T18:04:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 shutdown_connections 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0740662f0 0x7fe0740687a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0900c3bb0 0x7fe0901ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0900cc880 0x7fe0900c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 --2- 172.21.15.22:0/2861337486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0900cc120 0x7fe09014d6b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 >> 172.21.15.22:0/2861337486 conn(0x7fe09006a5f0 msgr2=0x7fe09006d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 shutdown_connections 2023-06-23T18:04:04.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.015+0000 7fe096ab5700 1 -- 172.21.15.22:0/2861337486 wait complete. 2023-06-23T18:04:04.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-06-23T18:04:04.042 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-06-23T18:04:04.043 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-06-23T18:04:04.044 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-06-23T18:04:04.045 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-06-23T18:04:04.047 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:04.047 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-23T18:04:04.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:04:04.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 Processor -- start 2023-06-23T18:04:04.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- start start 2023-06-23T18:04:04.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc878160420 0x7fc8780cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc878070040 0x7fc8780cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc878163ec0 con 0x7fc8780c0fe0 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780cc6f0 con 0x7fc878160420 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781641a0 con 0x7fc878070040 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc878160420 0x7fc8780cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc878160420 0x7fc8780cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44136/0 (socket says 172.21.15.22:44136) 2023-06-23T18:04:04.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 -- 172.21.15.22:0/4291753600 learned_addr learned my addr 172.21.15.22:0/4291753600 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:04.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 -- 172.21.15.22:0/4291753600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc878070040 msgr2=0x7fc8780cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:04.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 --2- 172.21.15.22:0/4291753600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc878070040 0x7fc8780cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 -- 172.21.15.22:0/4291753600 >> v1:172.21.15.169:6789/0 conn(0x7fc8780c0fe0 legacy=0x7fc8780c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:04.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 -- 172.21.15.22:0/4291753600 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc878164480 con 0x7fc878160420 2023-06-23T18:04:04.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc875d9b700 1 --2- 172.21.15.22:0/4291753600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878160420 0x7fc8780cbc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8780c85f0 tx=0x7fc87000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405d255e450d2fca server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87559a700 1 -- 172.21.15.22:0/4291753600 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000e070 con 0x7fc878160420 2023-06-23T18:04:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87559a700 1 -- 172.21.15.22:0/4291753600 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000b980 con 0x7fc878160420 2023-06-23T18:04:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87559a700 1 -- 172.21.15.22:0/4291753600 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000bd30 con 0x7fc878160420 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- 172.21.15.22:0/4291753600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878160420 msgr2=0x7fc8780cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 --2- 172.21.15.22:0/4291753600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878160420 0x7fc8780cbc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8780c85f0 tx=0x7fc87000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- 172.21.15.22:0/4291753600 shutdown_connections 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 --2- 172.21.15.22:0/4291753600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc878070040 0x7fc8780cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 --2- 172.21.15.22:0/4291753600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878160420 0x7fc8780cbc70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- 172.21.15.22:0/4291753600 >> 172.21.15.22:0/4291753600 conn(0x7fc87806a5f0 msgr2=0x7fc8780caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- 172.21.15.22:0/4291753600 shutdown_connections 2023-06-23T18:04:04.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.183+0000 7fc87759e700 1 -- 172.21.15.22:0/4291753600 wait complete. 2023-06-23T18:04:04.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 Processor -- start 2023-06-23T18:04:04.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 -- start start 2023-06-23T18:04:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780c0fe0 0x7fc8780c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc878160420 0x7fc8780c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780b0830 con 0x7fc878070040 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878161280 con 0x7fc878160420 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780c7030 con 0x7fc8780c0fe0 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44148/0 (socket says 172.21.15.22:44148) 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc87659c700 1 -- 172.21.15.22:0/902923466 learned_addr learned my addr 172.21.15.22:0/902923466 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc875d9b700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780c0fe0 0x7fc8780c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.187+0000 7fc876d9d700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc878160420 0x7fc8780c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 -- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780c0fe0 msgr2=0x7fc8780c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780c0fe0 0x7fc8780c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 -- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc878160420 msgr2=0x7fc8780c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc878160420 0x7fc8780c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 -- 172.21.15.22:0/902923466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009b30 con 0x7fc878070040 2023-06-23T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87659c700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c4e0 tx=0x7fc86000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860011790 con 0x7fc878070040 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860011910 con 0x7fc878070040 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8780c7310 con 0x7fc878070040 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860011cc0 con 0x7fc878070040 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8780c78d0 con 0x7fc878070040 2023-06-23T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8780b0970 con 0x7fc878070040 2023-06-23T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc86001c460 con 0x7fc878070040 2023-06-23T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.191+0000 7fc86effd700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 0x7fc8580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.195+0000 7fc875d9b700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 0x7fc8580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.195+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc860083e10 con 0x7fc878070040 2023-06-23T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.195+0000 7fc875d9b700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 0x7fc8580687a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8780c85f0 tx=0x7fc870009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.199+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc8780b0970 con 0x7fc878070040 2023-06-23T18:04:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.351+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc878164a40 con 0x7fc8580662f0 2023-06-23T18:04:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.371+0000 7fc86effd700 1 -- 172.21.15.22:0/902923466 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367249 (secure 0 0 0) 0x7fc870021070 con 0x7fc8580662f0 2023-06-23T18:04:04.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:04.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 msgr2=0x7fc8580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 0x7fc8580687a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8780c85f0 tx=0x7fc870009000 comp rx=0 tx=0).stop 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 msgr2=0x7fc8780b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c4e0 tx=0x7fc86000c7f0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 shutdown_connections 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc8580662f0 0x7fc8580687a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc878160420 0x7fc8780c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780c0fe0 0x7fc8780c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 --2- 172.21.15.22:0/902923466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc878070040 0x7fc8780b02f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 >> 172.21.15.22:0/902923466 conn(0x7fc87806a5f0 msgr2=0x7fc8780c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 shutdown_connections 2023-06-23T18:04:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.375+0000 7fc87759e700 1 -- 172.21.15.22:0/902923466 wait complete. 2023-06-23T18:04:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:04:04.404 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-06-23T18:04:03.615910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235208,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":3284992,"data_stored":2027694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004683"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915552+0000","last_change":"2023-06-23T18:03:41.789565+0000","last_active":"2023-06-23T18:03:42.915552+0000","last_peered":"2023-06-23T18:03:42.915552+0000","last_clean":"2023-06-23T18:03:42.915552+0000","last_became_active":"2023-06-23T18:03:41.787653+0000","last_became_peered":"2023-06-23T18:03:41.787653+0000","last_unstale":"2023-06-23T18:03:42.915552+0000","last_undegraded":"2023-06-23T18:03:42.915552+0000","last_fullsized":"2023-06-23T18:03:42.915552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922374+0000","last_change":"2023-06-23T18:03:37.377461+0000","last_active":"2023-06-23T18:03:42.922374+0000","last_peered":"2023-06-23T18:03:42.922374+0000","last_clean":"2023-06-23T18:03:42.922374+0000","last_became_active":"2023-06-23T18:03:37.377017+0000","last_became_peered":"2023-06-23T18:03:37.377017+0000","last_unstale":"2023-06-23T18:03:42.922374+0000","last_undegraded":"2023-06-23T18:03:42.922374+0000","last_fullsized":"2023-06-23T18:03:42.922374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157012+0000","last_change":"2023-06-23T18:03:37.756400+0000","last_active":"2023-06-23T18:03:42.157012+0000","last_peered":"2023-06-23T18:03:42.157012+0000","last_clean":"2023-06-23T18:03:42.157012+0000","last_became_active":"2023-06-23T18:03:37.755870+0000","last_became_peered":"2023-06-23T18:03:37.755870+0000","last_unstale":"2023-06-23T18:03:42.157012+0000","last_undegraded":"2023-06-23T18:03:42.157012+0000","last_fullsized":"2023-06-23T18:03:42.157012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:35:48.712508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157286+0000","last_change":"2023-06-23T18:03:41.784738+0000","last_active":"2023-06-23T18:03:42.157286+0000","last_peered":"2023-06-23T18:03:42.157286+0000","last_clean":"2023-06-23T18:03:42.157286+0000","last_became_active":"2023-06-23T18:03:41.784348+0000","last_became_peered":"2023-06-23T18:03:41.784348+0000","last_unstale":"2023-06-23T18:03:42.157286+0000","last_undegraded":"2023-06-23T18:03:42.157286+0000","last_fullsized":"2023-06-23T18:03:42.157286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:06:40.211756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920797+0000","last_change":"2023-06-23T18:03:37.364428+0000","last_active":"2023-06-23T18:03:42.920797+0000","last_peered":"2023-06-23T18:03:42.920797+0000","last_clean":"2023-06-23T18:03:42.920797+0000","last_became_active":"2023-06-23T18:03:37.364114+0000","last_became_peered":"2023-06-23T18:03:37.364114+0000","last_unstale":"2023-06-23T18:03:42.920797+0000","last_undegraded":"2023-06-23T18:03:42.920797+0000","last_fullsized":"2023-06-23T18:03:42.920797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:53.340210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917007+0000","last_change":"2023-06-23T18:03:37.761252+0000","last_active":"2023-06-23T18:03:42.917007+0000","last_peered":"2023-06-23T18:03:42.917007+0000","last_clean":"2023-06-23T18:03:42.917007+0000","last_became_active":"2023-06-23T18:03:37.760991+0000","last_became_peered":"2023-06-23T18:03:37.760991+0000","last_unstale":"2023-06-23T18:03:42.917007+0000","last_undegraded":"2023-06-23T18:03:42.917007+0000","last_fullsized":"2023-06-23T18:03:42.917007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:26.556145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922896+0000","last_change":"2023-06-23T18:03:41.789561+0000","last_active":"2023-06-23T18:03:42.922896+0000","last_peered":"2023-06-23T18:03:42.922896+0000","last_clean":"2023-06-23T18:03:42.922896+0000","last_became_active":"2023-06-23T18:03:41.788768+0000","last_became_peered":"2023-06-23T18:03:41.788768+0000","last_unstale":"2023-06-23T18:03:42.922896+0000","last_undegraded":"2023-06-23T18:03:42.922896+0000","last_fullsized":"2023-06-23T18:03:42.922896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924155+0000","last_change":"2023-06-23T18:03:37.377124+0000","last_active":"2023-06-23T18:03:42.924155+0000","last_peered":"2023-06-23T18:03:42.924155+0000","last_clean":"2023-06-23T18:03:42.924155+0000","last_became_active":"2023-06-23T18:03:37.376829+0000","last_became_peered":"2023-06-23T18:03:37.376829+0000","last_unstale":"2023-06-23T18:03:42.924155+0000","last_undegraded":"2023-06-23T18:03:42.924155+0000","last_fullsized":"2023-06-23T18:03:42.924155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157618+0000","last_change":"2023-06-23T18:03:37.754592+0000","last_active":"2023-06-23T18:03:42.157618+0000","last_peered":"2023-06-23T18:03:42.157618+0000","last_clean":"2023-06-23T18:03:42.157618+0000","last_became_active":"2023-06-23T18:03:37.754279+0000","last_became_peered":"2023-06-23T18:03:37.754279+0000","last_unstale":"2023-06-23T18:03:42.157618+0000","last_undegraded":"2023-06-23T18:03:42.157618+0000","last_fullsized":"2023-06-23T18:03:42.157618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:05:29.058510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920916+0000","last_change":"2023-06-23T18:03:41.787327+0000","last_active":"2023-06-23T18:03:42.920916+0000","last_peered":"2023-06-23T18:03:42.920916+0000","last_clean":"2023-06-23T18:03:42.920916+0000","last_became_active":"2023-06-23T18:03:41.786664+0000","last_became_peered":"2023-06-23T18:03:41.786664+0000","last_unstale":"2023-06-23T18:03:42.920916+0000","last_undegraded":"2023-06-23T18:03:42.920916+0000","last_fullsized":"2023-06-23T18:03:42.920916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922619+0000","last_change":"2023-06-23T18:03:37.377468+0000","last_active":"2023-06-23T18:03:42.922619+0000","last_peered":"2023-06-23T18:03:42.922619+0000","last_clean":"2023-06-23T18:03:42.922619+0000","last_became_active":"2023-06-23T18:03:37.376653+0000","last_became_peered":"2023-06-23T18:03:37.376653+0000","last_unstale":"2023-06-23T18:03:42.922619+0000","last_undegraded":"2023-06-23T18:03:42.922619+0000","last_fullsized":"2023-06-23T18:03:42.922619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157926+0000","last_change":"2023-06-23T18:03:37.761926+0000","last_active":"2023-06-23T18:03:42.157926+0000","last_peered":"2023-06-23T18:03:42.157926+0000","last_clean":"2023-06-23T18:03:42.157926+0000","last_became_active":"2023-06-23T18:03:37.761621+0000","last_became_peered":"2023-06-23T18:03:37.761621+0000","last_unstale":"2023-06-23T18:03:42.157926+0000","last_undegraded":"2023-06-23T18:03:42.157926+0000","last_fullsized":"2023-06-23T18:03:42.157926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:20.322939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918838+0000","last_change":"2023-06-23T18:03:41.782917+0000","last_active":"2023-06-23T18:03:42.918838+0000","last_peered":"2023-06-23T18:03:42.918838+0000","last_clean":"2023-06-23T18:03:42.918838+0000","last_became_active":"2023-06-23T18:03:41.782516+0000","last_became_peered":"2023-06-23T18:03:41.782516+0000","last_unstale":"2023-06-23T18:03:42.918838+0000","last_undegraded":"2023-06-23T18:03:42.918838+0000","last_fullsized":"2023-06-23T18:03:42.918838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920996+0000","last_change":"2023-06-23T18:03:37.379511+0000","last_active":"2023-06-23T18:03:42.920996+0000","last_peered":"2023-06-23T18:03:42.920996+0000","last_clean":"2023-06-23T18:03:42.920996+0000","last_became_active":"2023-06-23T18:03:37.379252+0000","last_became_peered":"2023-06-23T18:03:37.379252+0000","last_unstale":"2023-06-23T18:03:42.920996+0000","last_undegraded":"2023-06-23T18:03:42.920996+0000","last_fullsized":"2023-06-23T18:03:42.920996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:58.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153021+0000","last_change":"2023-06-23T18:03:37.768862+0000","last_active":"2023-06-23T18:03:42.153021+0000","last_peered":"2023-06-23T18:03:42.153021+0000","last_clean":"2023-06-23T18:03:42.153021+0000","last_became_active":"2023-06-23T18:03:37.768610+0000","last_became_peered":"2023-06-23T18:03:37.768610+0000","last_unstale":"2023-06-23T18:03:42.153021+0000","last_undegraded":"2023-06-23T18:03:42.153021+0000","last_fullsized":"2023-06-23T18:03:42.153021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:10:03.453386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158181+0000","last_change":"2023-06-23T18:03:41.781852+0000","last_active":"2023-06-23T18:03:42.158181+0000","last_peered":"2023-06-23T18:03:42.158181+0000","last_clean":"2023-06-23T18:03:42.158181+0000","last_became_active":"2023-06-23T18:03:41.781321+0000","last_became_peered":"2023-06-23T18:03:41.781321+0000","last_unstale":"2023-06-23T18:03:42.158181+0000","last_undegraded":"2023-06-23T18:03:42.158181+0000","last_fullsized":"2023-06-23T18:03:42.158181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:21:44.946317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157942+0000","last_change":"2023-06-23T18:03:37.372568+0000","last_active":"2023-06-23T18:03:42.157942+0000","last_peered":"2023-06-23T18:03:42.157942+0000","last_clean":"2023-06-23T18:03:42.157942+0000","last_became_active":"2023-06-23T18:03:37.372256+0000","last_became_peered":"2023-06-23T18:03:37.372256+0000","last_unstale":"2023-06-23T18:03:42.157942+0000","last_undegraded":"2023-06-23T18:03:42.157942+0000","last_fullsized":"2023-06-23T18:03:42.157942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:01:27.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919391+0000","last_change":"2023-06-23T18:03:37.761387+0000","last_active":"2023-06-23T18:03:42.919391+0000","last_peered":"2023-06-23T18:03:42.919391+0000","last_clean":"2023-06-23T18:03:42.919391+0000","last_became_active":"2023-06-23T18:03:37.758859+0000","last_became_peered":"2023-06-23T18:03:37.758859+0000","last_unstale":"2023-06-23T18:03:42.919391+0000","last_undegraded":"2023-06-23T18:03:42.919391+0000","last_fullsized":"2023-06-23T18:03:42.919391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919777+0000","last_change":"2023-06-23T18:03:41.782177+0000","last_active":"2023-06-23T18:03:42.919777+0000","last_peered":"2023-06-23T18:03:42.919777+0000","last_clean":"2023-06-23T18:03:42.919777+0000","last_became_active":"2023-06-23T18:03:41.781457+0000","last_became_peered":"2023-06-23T18:03:41.781457+0000","last_unstale":"2023-06-23T18:03:42.919777+0000","last_undegraded":"2023-06-23T18:03:42.919777+0000","last_fullsized":"2023-06-23T18:03:42.919777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:38:17.449084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916697+0000","last_change":"2023-06-23T18:03:37.367776+0000","last_active":"2023-06-23T18:03:42.916697+0000","last_peered":"2023-06-23T18:03:42.916697+0000","last_clean":"2023-06-23T18:03:42.916697+0000","last_became_active":"2023-06-23T18:03:37.367288+0000","last_became_peered":"2023-06-23T18:03:37.367288+0000","last_unstale":"2023-06-23T18:03:42.916697+0000","last_undegraded":"2023-06-23T18:03:42.916697+0000","last_fullsized":"2023-06-23T18:03:42.916697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:46.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922187+0000","last_change":"2023-06-23T18:03:37.769820+0000","last_active":"2023-06-23T18:03:42.922187+0000","last_peered":"2023-06-23T18:03:42.922187+0000","last_clean":"2023-06-23T18:03:42.922187+0000","last_became_active":"2023-06-23T18:03:37.769571+0000","last_became_peered":"2023-06-23T18:03:37.769571+0000","last_unstale":"2023-06-23T18:03:42.922187+0000","last_undegraded":"2023-06-23T18:03:42.922187+0000","last_fullsized":"2023-06-23T18:03:42.922187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921931+0000","last_change":"2023-06-23T18:03:41.783182+0000","last_active":"2023-06-23T18:03:42.921931+0000","last_peered":"2023-06-23T18:03:42.921931+0000","last_clean":"2023-06-23T18:03:42.921931+0000","last_became_active":"2023-06-23T18:03:41.782753+0000","last_became_peered":"2023-06-23T18:03:41.782753+0000","last_unstale":"2023-06-23T18:03:42.921931+0000","last_undegraded":"2023-06-23T18:03:42.921931+0000","last_fullsized":"2023-06-23T18:03:42.921931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916193+0000","last_change":"2023-06-23T18:03:37.378552+0000","last_active":"2023-06-23T18:03:42.916193+0000","last_peered":"2023-06-23T18:03:42.916193+0000","last_clean":"2023-06-23T18:03:42.916193+0000","last_became_active":"2023-06-23T18:03:37.378264+0000","last_became_peered":"2023-06-23T18:03:37.378264+0000","last_unstale":"2023-06-23T18:03:42.916193+0000","last_undegraded":"2023-06-23T18:03:42.916193+0000","last_fullsized":"2023-06-23T18:03:42.916193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:04:27.977965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920059+0000","last_change":"2023-06-23T18:03:37.755621+0000","last_active":"2023-06-23T18:03:42.920059+0000","last_peered":"2023-06-23T18:03:42.920059+0000","last_clean":"2023-06-23T18:03:42.920059+0000","last_became_active":"2023-06-23T18:03:37.755341+0000","last_became_peered":"2023-06-23T18:03:37.755341+0000","last_unstale":"2023-06-23T18:03:42.920059+0000","last_undegraded":"2023-06-23T18:03:42.920059+0000","last_fullsized":"2023-06-23T18:03:42.920059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:11:22.805985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915863+0000","last_change":"2023-06-23T18:03:37.758569+0000","last_active":"2023-06-23T18:03:42.915863+0000","last_peered":"2023-06-23T18:03:42.915863+0000","last_clean":"2023-06-23T18:03:42.915863+0000","last_became_active":"2023-06-23T18:03:37.758229+0000","last_became_peered":"2023-06-23T18:03:37.758229+0000","last_unstale":"2023-06-23T18:03:42.915863+0000","last_undegraded":"2023-06-23T18:03:42.915863+0000","last_fullsized":"2023-06-23T18:03:42.915863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:56:28.950360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922336+0000","last_change":"2023-06-23T18:03:37.377384+0000","last_active":"2023-06-23T18:03:42.922336+0000","last_peered":"2023-06-23T18:03:42.922336+0000","last_clean":"2023-06-23T18:03:42.922336+0000","last_became_active":"2023-06-23T18:03:37.376940+0000","last_became_peered":"2023-06-23T18:03:37.376940+0000","last_unstale":"2023-06-23T18:03:42.922336+0000","last_undegraded":"2023-06-23T18:03:42.922336+0000","last_fullsized":"2023-06-23T18:03:42.922336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919105+0000","last_change":"2023-06-23T18:03:41.787485+0000","last_active":"2023-06-23T18:03:42.919105+0000","last_peered":"2023-06-23T18:03:42.919105+0000","last_clean":"2023-06-23T18:03:42.919105+0000","last_became_active":"2023-06-23T18:03:41.787017+0000","last_became_peered":"2023-06-23T18:03:41.787017+0000","last_unstale":"2023-06-23T18:03:42.919105+0000","last_undegraded":"2023-06-23T18:03:42.919105+0000","last_fullsized":"2023-06-23T18:03:42.919105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:48:29.879084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916767+0000","last_change":"2023-06-23T18:03:37.768665+0000","last_active":"2023-06-23T18:03:42.916767+0000","last_peered":"2023-06-23T18:03:42.916767+0000","last_clean":"2023-06-23T18:03:42.916767+0000","last_became_active":"2023-06-23T18:03:37.768373+0000","last_became_peered":"2023-06-23T18:03:37.768373+0000","last_unstale":"2023-06-23T18:03:42.916767+0000","last_undegraded":"2023-06-23T18:03:42.916767+0000","last_fullsized":"2023-06-23T18:03:42.916767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:03:33.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923756+0000","last_change":"2023-06-23T18:03:37.376353+0000","last_active":"2023-06-23T18:03:42.923756+0000","last_peered":"2023-06-23T18:03:42.923756+0000","last_clean":"2023-06-23T18:03:42.923756+0000","last_became_active":"2023-06-23T18:03:37.375931+0000","last_became_peered":"2023-06-23T18:03:37.375931+0000","last_unstale":"2023-06-23T18:03:42.923756+0000","last_undegraded":"2023-06-23T18:03:42.923756+0000","last_fullsized":"2023-06-23T18:03:42.923756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:22.517180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920338+0000","last_change":"2023-06-23T18:03:41.787494+0000","last_active":"2023-06-23T18:03:42.920338+0000","last_peered":"2023-06-23T18:03:42.920338+0000","last_clean":"2023-06-23T18:03:42.920338+0000","last_became_active":"2023-06-23T18:03:41.787099+0000","last_became_peered":"2023-06-23T18:03:41.787099+0000","last_unstale":"2023-06-23T18:03:42.920338+0000","last_undegraded":"2023-06-23T18:03:42.920338+0000","last_fullsized":"2023-06-23T18:03:42.920338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922054+0000","last_change":"2023-06-23T18:03:37.377640+0000","last_active":"2023-06-23T18:03:42.922054+0000","last_peered":"2023-06-23T18:03:42.922054+0000","last_clean":"2023-06-23T18:03:42.922054+0000","last_became_active":"2023-06-23T18:03:37.377010+0000","last_became_peered":"2023-06-23T18:03:37.377010+0000","last_unstale":"2023-06-23T18:03:42.922054+0000","last_undegraded":"2023-06-23T18:03:42.922054+0000","last_fullsized":"2023-06-23T18:03:42.922054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917168+0000","last_change":"2023-06-23T18:03:37.756104+0000","last_active":"2023-06-23T18:03:42.917168+0000","last_peered":"2023-06-23T18:03:42.917168+0000","last_clean":"2023-06-23T18:03:42.917168+0000","last_became_active":"2023-06-23T18:03:37.755767+0000","last_became_peered":"2023-06-23T18:03:37.755767+0000","last_unstale":"2023-06-23T18:03:42.917168+0000","last_undegraded":"2023-06-23T18:03:42.917168+0000","last_fullsized":"2023-06-23T18:03:42.917168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:22:59.783746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917308+0000","last_change":"2023-06-23T18:03:41.785136+0000","last_active":"2023-06-23T18:03:42.917308+0000","last_peered":"2023-06-23T18:03:42.917308+0000","last_clean":"2023-06-23T18:03:42.917308+0000","last_became_active":"2023-06-23T18:03:41.784711+0000","last_became_peered":"2023-06-23T18:03:41.784711+0000","last_unstale":"2023-06-23T18:03:42.917308+0000","last_undegraded":"2023-06-23T18:03:42.917308+0000","last_fullsized":"2023-06-23T18:03:42.917308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:47.124821+0000","last_change":"2023-06-23T18:03:37.376017+0000","last_active":"2023-06-23T18:03:47.124821+0000","last_peered":"2023-06-23T18:03:47.124821+0000","last_clean":"2023-06-23T18:03:47.124821+0000","last_became_active":"2023-06-23T18:03:37.375728+0000","last_became_peered":"2023-06-23T18:03:37.375728+0000","last_unstale":"2023-06-23T18:03:47.124821+0000","last_undegraded":"2023-06-23T18:03:47.124821+0000","last_fullsized":"2023-06-23T18:03:47.124821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:57.309841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158169+0000","last_change":"2023-06-23T18:03:37.752163+0000","last_active":"2023-06-23T18:03:42.158169+0000","last_peered":"2023-06-23T18:03:42.158169+0000","last_clean":"2023-06-23T18:03:42.158169+0000","last_became_active":"2023-06-23T18:03:37.751393+0000","last_became_peered":"2023-06-23T18:03:37.751393+0000","last_unstale":"2023-06-23T18:03:42.158169+0000","last_undegraded":"2023-06-23T18:03:42.158169+0000","last_fullsized":"2023-06-23T18:03:42.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:54:13.905688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916838+0000","last_change":"2023-06-23T18:03:41.790305+0000","last_active":"2023-06-23T18:03:42.916838+0000","last_peered":"2023-06-23T18:03:42.916838+0000","last_clean":"2023-06-23T18:03:42.916838+0000","last_became_active":"2023-06-23T18:03:41.787857+0000","last_became_peered":"2023-06-23T18:03:41.787857+0000","last_unstale":"2023-06-23T18:03:42.916838+0000","last_undegraded":"2023-06-23T18:03:42.916838+0000","last_fullsized":"2023-06-23T18:03:42.916838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:54.379340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157126+0000","last_change":"2023-06-23T18:03:37.751336+0000","last_active":"2023-06-23T18:03:42.157126+0000","last_peered":"2023-06-23T18:03:42.157126+0000","last_clean":"2023-06-23T18:03:42.157126+0000","last_became_active":"2023-06-23T18:03:37.750981+0000","last_became_peered":"2023-06-23T18:03:37.750981+0000","last_unstale":"2023-06-23T18:03:42.157126+0000","last_undegraded":"2023-06-23T18:03:42.157126+0000","last_fullsized":"2023-06-23T18:03:42.157126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:17:15.424492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920331+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:03:42.920331+0000","last_peered":"2023-06-23T18:03:42.920331+0000","last_clean":"2023-06-23T18:03:42.920331+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:03:42.920331+0000","last_undegraded":"2023-06-23T18:03:42.920331+0000","last_fullsized":"2023-06-23T18:03:42.920331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:49:09.922836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922412+0000","last_change":"2023-06-23T18:03:37.375755+0000","last_active":"2023-06-23T18:03:42.922412+0000","last_peered":"2023-06-23T18:03:42.922412+0000","last_clean":"2023-06-23T18:03:42.922412+0000","last_became_active":"2023-06-23T18:03:37.375421+0000","last_became_peered":"2023-06-23T18:03:37.375421+0000","last_unstale":"2023-06-23T18:03:42.922412+0000","last_undegraded":"2023-06-23T18:03:42.922412+0000","last_fullsized":"2023-06-23T18:03:42.922412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915864+0000","last_change":"2023-06-23T18:03:41.785345+0000","last_active":"2023-06-23T18:03:42.915864+0000","last_peered":"2023-06-23T18:03:42.915864+0000","last_clean":"2023-06-23T18:03:42.915864+0000","last_became_active":"2023-06-23T18:03:41.784105+0000","last_became_peered":"2023-06-23T18:03:41.784105+0000","last_unstale":"2023-06-23T18:03:42.915864+0000","last_undegraded":"2023-06-23T18:03:42.915864+0000","last_fullsized":"2023-06-23T18:03:42.915864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:07:51.445920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915986+0000","last_change":"2023-06-23T18:03:37.768794+0000","last_active":"2023-06-23T18:03:42.915986+0000","last_peered":"2023-06-23T18:03:42.915986+0000","last_clean":"2023-06-23T18:03:42.915986+0000","last_became_active":"2023-06-23T18:03:37.768517+0000","last_became_peered":"2023-06-23T18:03:37.768517+0000","last_unstale":"2023-06-23T18:03:42.915986+0000","last_undegraded":"2023-06-23T18:03:42.915986+0000","last_fullsized":"2023-06-23T18:03:42.915986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:53.245906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920435+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:03:42.920435+0000","last_peered":"2023-06-23T18:03:42.920435+0000","last_clean":"2023-06-23T18:03:42.920435+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:03:42.920435+0000","last_undegraded":"2023-06-23T18:03:42.920435+0000","last_fullsized":"2023-06-23T18:03:42.920435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921748+0000","last_change":"2023-06-23T18:03:37.376156+0000","last_active":"2023-06-23T18:03:42.921748+0000","last_peered":"2023-06-23T18:03:42.921748+0000","last_clean":"2023-06-23T18:03:42.921748+0000","last_became_active":"2023-06-23T18:03:37.375870+0000","last_became_peered":"2023-06-23T18:03:37.375870+0000","last_unstale":"2023-06-23T18:03:42.921748+0000","last_undegraded":"2023-06-23T18:03:42.921748+0000","last_fullsized":"2023-06-23T18:03:42.921748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156557+0000","last_change":"2023-06-23T18:03:41.783782+0000","last_active":"2023-06-23T18:03:42.156557+0000","last_peered":"2023-06-23T18:03:42.156557+0000","last_clean":"2023-06-23T18:03:42.156557+0000","last_became_active":"2023-06-23T18:03:41.783336+0000","last_became_peered":"2023-06-23T18:03:41.783336+0000","last_unstale":"2023-06-23T18:03:42.156557+0000","last_undegraded":"2023-06-23T18:03:42.156557+0000","last_fullsized":"2023-06-23T18:03:42.156557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:55.511609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920320+0000","last_change":"2023-06-23T18:03:37.755707+0000","last_active":"2023-06-23T18:03:42.920320+0000","last_peered":"2023-06-23T18:03:42.920320+0000","last_clean":"2023-06-23T18:03:42.920320+0000","last_became_active":"2023-06-23T18:03:37.755413+0000","last_became_peered":"2023-06-23T18:03:37.755413+0000","last_unstale":"2023-06-23T18:03:42.920320+0000","last_undegraded":"2023-06-23T18:03:42.920320+0000","last_fullsized":"2023-06-23T18:03:42.920320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922407+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:03:42.922407+0000","last_peered":"2023-06-23T18:03:42.922407+0000","last_clean":"2023-06-23T18:03:42.922407+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:03:42.922407+0000","last_undegraded":"2023-06-23T18:03:42.922407+0000","last_fullsized":"2023-06-23T18:03:42.922407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:55.340944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918354+0000","last_change":"2023-06-23T18:03:37.370271+0000","last_active":"2023-06-23T18:03:42.918354+0000","last_peered":"2023-06-23T18:03:42.918354+0000","last_clean":"2023-06-23T18:03:42.918354+0000","last_became_active":"2023-06-23T18:03:37.369994+0000","last_became_peered":"2023-06-23T18:03:37.369994+0000","last_unstale":"2023-06-23T18:03:42.918354+0000","last_undegraded":"2023-06-23T18:03:42.918354+0000","last_fullsized":"2023-06-23T18:03:42.918354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:36.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920193+0000","last_change":"2023-06-23T18:03:41.793568+0000","last_active":"2023-06-23T18:03:42.920193+0000","last_peered":"2023-06-23T18:03:42.920193+0000","last_clean":"2023-06-23T18:03:42.920193+0000","last_became_active":"2023-06-23T18:03:41.793190+0000","last_became_peered":"2023-06-23T18:03:41.793190+0000","last_unstale":"2023-06-23T18:03:42.920193+0000","last_undegraded":"2023-06-23T18:03:42.920193+0000","last_fullsized":"2023-06-23T18:03:42.920193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:35.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917232+0000","last_change":"2023-06-23T18:03:37.764627+0000","last_active":"2023-06-23T18:03:42.917232+0000","last_peered":"2023-06-23T18:03:42.917232+0000","last_clean":"2023-06-23T18:03:42.917232+0000","last_became_active":"2023-06-23T18:03:37.764255+0000","last_became_peered":"2023-06-23T18:03:37.764255+0000","last_unstale":"2023-06-23T18:03:42.917232+0000","last_undegraded":"2023-06-23T18:03:42.917232+0000","last_fullsized":"2023-06-23T18:03:42.917232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918569+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:03:42.918569+0000","last_peered":"2023-06-23T18:03:42.918569+0000","last_clean":"2023-06-23T18:03:42.918569+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:03:42.918569+0000","last_undegraded":"2023-06-23T18:03:42.918569+0000","last_fullsized":"2023-06-23T18:03:42.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:25.233307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924293+0000","last_change":"2023-06-23T18:03:37.371127+0000","last_active":"2023-06-23T18:03:42.924293+0000","last_peered":"2023-06-23T18:03:42.924293+0000","last_clean":"2023-06-23T18:03:42.924293+0000","last_became_active":"2023-06-23T18:03:37.370810+0000","last_became_peered":"2023-06-23T18:03:37.370810+0000","last_unstale":"2023-06-23T18:03:42.924293+0000","last_undegraded":"2023-06-23T18:03:42.924293+0000","last_fullsized":"2023-06-23T18:03:42.924293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:42.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919827+0000","last_change":"2023-06-23T18:03:41.791645+0000","last_active":"2023-06-23T18:03:42.919827+0000","last_peered":"2023-06-23T18:03:42.919827+0000","last_clean":"2023-06-23T18:03:42.919827+0000","last_became_active":"2023-06-23T18:03:41.791276+0000","last_became_peered":"2023-06-23T18:03:41.791276+0000","last_unstale":"2023-06-23T18:03:42.919827+0000","last_undegraded":"2023-06-23T18:03:42.919827+0000","last_fullsized":"2023-06-23T18:03:42.919827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:24.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918088+0000","last_change":"2023-06-23T18:03:37.754565+0000","last_active":"2023-06-23T18:03:42.918088+0000","last_peered":"2023-06-23T18:03:42.918088+0000","last_clean":"2023-06-23T18:03:42.918088+0000","last_became_active":"2023-06-23T18:03:37.754281+0000","last_became_peered":"2023-06-23T18:03:37.754281+0000","last_unstale":"2023-06-23T18:03:42.918088+0000","last_undegraded":"2023-06-23T18:03:42.918088+0000","last_fullsized":"2023-06-23T18:03:42.918088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158267+0000","last_change":"2023-06-23T18:03:37.381149+0000","last_active":"2023-06-23T18:03:42.158267+0000","last_peered":"2023-06-23T18:03:42.158267+0000","last_clean":"2023-06-23T18:03:42.158267+0000","last_became_active":"2023-06-23T18:03:37.380831+0000","last_became_peered":"2023-06-23T18:03:37.380831+0000","last_unstale":"2023-06-23T18:03:42.158267+0000","last_undegraded":"2023-06-23T18:03:42.158267+0000","last_fullsized":"2023-06-23T18:03:42.158267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:40:27.306991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923558+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:03:42.923558+0000","last_peered":"2023-06-23T18:03:42.923558+0000","last_clean":"2023-06-23T18:03:42.923558+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:03:42.923558+0000","last_undegraded":"2023-06-23T18:03:42.923558+0000","last_fullsized":"2023-06-23T18:03:42.923558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921567+0000","last_change":"2023-06-23T18:03:41.787633+0000","last_active":"2023-06-23T18:03:42.921567+0000","last_peered":"2023-06-23T18:03:42.921567+0000","last_clean":"2023-06-23T18:03:42.921567+0000","last_became_active":"2023-06-23T18:03:41.786906+0000","last_became_peered":"2023-06-23T18:03:41.786906+0000","last_unstale":"2023-06-23T18:03:42.921567+0000","last_undegraded":"2023-06-23T18:03:42.921567+0000","last_fullsized":"2023-06-23T18:03:42.921567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919645+0000","last_change":"2023-06-23T18:03:37.764949+0000","last_active":"2023-06-23T18:03:42.919645+0000","last_peered":"2023-06-23T18:03:42.919645+0000","last_clean":"2023-06-23T18:03:42.919645+0000","last_became_active":"2023-06-23T18:03:37.764358+0000","last_became_peered":"2023-06-23T18:03:37.764358+0000","last_unstale":"2023-06-23T18:03:42.919645+0000","last_undegraded":"2023-06-23T18:03:42.919645+0000","last_fullsized":"2023-06-23T18:03:42.919645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916086+0000","last_change":"2023-06-23T18:03:37.371283+0000","last_active":"2023-06-23T18:03:42.916086+0000","last_peered":"2023-06-23T18:03:42.916086+0000","last_clean":"2023-06-23T18:03:42.916086+0000","last_became_active":"2023-06-23T18:03:37.370358+0000","last_became_peered":"2023-06-23T18:03:37.370358+0000","last_unstale":"2023-06-23T18:03:42.916086+0000","last_undegraded":"2023-06-23T18:03:42.916086+0000","last_fullsized":"2023-06-23T18:03:42.916086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152204+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:03:42.152204+0000","last_peered":"2023-06-23T18:03:42.152204+0000","last_clean":"2023-06-23T18:03:42.152204+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:03:42.152204+0000","last_undegraded":"2023-06-23T18:03:42.152204+0000","last_fullsized":"2023-06-23T18:03:42.152204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:44.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152495+0000","last_change":"2023-06-23T18:03:41.787323+0000","last_active":"2023-06-23T18:03:42.152495+0000","last_peered":"2023-06-23T18:03:42.152495+0000","last_clean":"2023-06-23T18:03:42.152495+0000","last_became_active":"2023-06-23T18:03:41.786742+0000","last_became_peered":"2023-06-23T18:03:41.786742+0000","last_unstale":"2023-06-23T18:03:42.152495+0000","last_undegraded":"2023-06-23T18:03:42.152495+0000","last_fullsized":"2023-06-23T18:03:42.152495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:39:47.931343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922227+0000","last_change":"2023-06-23T18:03:37.743506+0000","last_active":"2023-06-23T18:03:42.922227+0000","last_peered":"2023-06-23T18:03:42.922227+0000","last_clean":"2023-06-23T18:03:42.922227+0000","last_became_active":"2023-06-23T18:03:37.743222+0000","last_became_peered":"2023-06-23T18:03:37.743222+0000","last_unstale":"2023-06-23T18:03:42.922227+0000","last_undegraded":"2023-06-23T18:03:42.922227+0000","last_fullsized":"2023-06-23T18:03:42.922227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:43.133522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920271+0000","last_change":"2023-06-23T18:03:37.375782+0000","last_active":"2023-06-23T18:03:42.920271+0000","last_peered":"2023-06-23T18:03:42.920271+0000","last_clean":"2023-06-23T18:03:42.920271+0000","last_became_active":"2023-06-23T18:03:37.375451+0000","last_became_peered":"2023-06-23T18:03:37.375451+0000","last_unstale":"2023-06-23T18:03:42.920271+0000","last_undegraded":"2023-06-23T18:03:42.920271+0000","last_fullsized":"2023-06-23T18:03:42.920271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:35.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921282+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:03:42.921282+0000","last_peered":"2023-06-23T18:03:42.921282+0000","last_clean":"2023-06-23T18:03:42.921282+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:03:42.921282+0000","last_undegraded":"2023-06-23T18:03:42.921282+0000","last_fullsized":"2023-06-23T18:03:42.921282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922998+0000","last_change":"2023-06-23T18:03:41.788944+0000","last_active":"2023-06-23T18:03:42.922998+0000","last_peered":"2023-06-23T18:03:42.922998+0000","last_clean":"2023-06-23T18:03:42.922998+0000","last_became_active":"2023-06-23T18:03:41.785916+0000","last_became_peered":"2023-06-23T18:03:41.785916+0000","last_unstale":"2023-06-23T18:03:42.922998+0000","last_undegraded":"2023-06-23T18:03:42.922998+0000","last_fullsized":"2023-06-23T18:03:42.922998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919919+0000","last_change":"2023-06-23T18:03:37.750749+0000","last_active":"2023-06-23T18:03:42.919919+0000","last_peered":"2023-06-23T18:03:42.919919+0000","last_clean":"2023-06-23T18:03:42.919919+0000","last_became_active":"2023-06-23T18:03:37.750424+0000","last_became_peered":"2023-06-23T18:03:37.750424+0000","last_unstale":"2023-06-23T18:03:42.919919+0000","last_undegraded":"2023-06-23T18:03:42.919919+0000","last_fullsized":"2023-06-23T18:03:42.919919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:10.920865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917678+0000","last_change":"2023-06-23T18:03:37.377053+0000","last_active":"2023-06-23T18:03:42.917678+0000","last_peered":"2023-06-23T18:03:42.917678+0000","last_clean":"2023-06-23T18:03:42.917678+0000","last_became_active":"2023-06-23T18:03:37.376770+0000","last_became_peered":"2023-06-23T18:03:37.376770+0000","last_unstale":"2023-06-23T18:03:42.917678+0000","last_undegraded":"2023-06-23T18:03:42.917678+0000","last_fullsized":"2023-06-23T18:03:42.917678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923092+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:03:42.923092+0000","last_peered":"2023-06-23T18:03:42.923092+0000","last_clean":"2023-06-23T18:03:42.923092+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:03:42.923092+0000","last_undegraded":"2023-06-23T18:03:42.923092+0000","last_fullsized":"2023-06-23T18:03:42.923092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:00.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921588+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:03:42.921588+0000","last_peered":"2023-06-23T18:03:42.921588+0000","last_clean":"2023-06-23T18:03:42.921588+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:03:42.921588+0000","last_undegraded":"2023-06-23T18:03:42.921588+0000","last_fullsized":"2023-06-23T18:03:42.921588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921628+0000","last_change":"2023-06-23T18:03:41.792372+0000","last_active":"2023-06-23T18:03:42.921628+0000","last_peered":"2023-06-23T18:03:42.921628+0000","last_clean":"2023-06-23T18:03:42.921628+0000","last_became_active":"2023-06-23T18:03:41.791937+0000","last_became_peered":"2023-06-23T18:03:41.791937+0000","last_unstale":"2023-06-23T18:03:42.921628+0000","last_undegraded":"2023-06-23T18:03:42.921628+0000","last_fullsized":"2023-06-23T18:03:42.921628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:26.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919683+0000","last_change":"2023-06-23T18:03:37.764721+0000","last_active":"2023-06-23T18:03:42.919683+0000","last_peered":"2023-06-23T18:03:42.919683+0000","last_clean":"2023-06-23T18:03:42.919683+0000","last_became_active":"2023-06-23T18:03:37.755578+0000","last_became_peered":"2023-06-23T18:03:37.755578+0000","last_unstale":"2023-06-23T18:03:42.919683+0000","last_undegraded":"2023-06-23T18:03:42.919683+0000","last_fullsized":"2023-06-23T18:03:42.919683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916131+0000","last_change":"2023-06-23T18:03:37.369824+0000","last_active":"2023-06-23T18:03:42.916131+0000","last_peered":"2023-06-23T18:03:42.916131+0000","last_clean":"2023-06-23T18:03:42.916131+0000","last_became_active":"2023-06-23T18:03:37.369094+0000","last_became_peered":"2023-06-23T18:03:37.369094+0000","last_unstale":"2023-06-23T18:03:42.916131+0000","last_undegraded":"2023-06-23T18:03:42.916131+0000","last_fullsized":"2023-06-23T18:03:42.916131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:53.895398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159093+0000","last_change":"2023-06-23T18:03:41.789799+0000","last_active":"2023-06-23T18:03:42.159093+0000","last_peered":"2023-06-23T18:03:42.159093+0000","last_clean":"2023-06-23T18:03:42.159093+0000","last_became_active":"2023-06-23T18:03:41.789426+0000","last_became_peered":"2023-06-23T18:03:41.789426+0000","last_unstale":"2023-06-23T18:03:42.159093+0000","last_undegraded":"2023-06-23T18:03:42.159093+0000","last_fullsized":"2023-06-23T18:03:42.159093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:36:43.668342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159103+0000","last_change":"2023-06-23T18:03:37.751763+0000","last_active":"2023-06-23T18:03:42.159103+0000","last_peered":"2023-06-23T18:03:42.159103+0000","last_clean":"2023-06-23T18:03:42.159103+0000","last_became_active":"2023-06-23T18:03:37.751310+0000","last_became_peered":"2023-06-23T18:03:37.751310+0000","last_unstale":"2023-06-23T18:03:42.159103+0000","last_undegraded":"2023-06-23T18:03:42.159103+0000","last_fullsized":"2023-06-23T18:03:42.159103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:00.581207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157189+0000","last_change":"2023-06-23T18:03:37.373425+0000","last_active":"2023-06-23T18:03:42.157189+0000","last_peered":"2023-06-23T18:03:42.157189+0000","last_clean":"2023-06-23T18:03:42.157189+0000","last_became_active":"2023-06-23T18:03:37.373154+0000","last_became_peered":"2023-06-23T18:03:37.373154+0000","last_unstale":"2023-06-23T18:03:42.157189+0000","last_undegraded":"2023-06-23T18:03:42.157189+0000","last_fullsized":"2023-06-23T18:03:42.157189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:33.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922639+0000","last_change":"2023-06-23T18:03:41.792127+0000","last_active":"2023-06-23T18:03:42.922639+0000","last_peered":"2023-06-23T18:03:42.922639+0000","last_clean":"2023-06-23T18:03:42.922639+0000","last_became_active":"2023-06-23T18:03:41.789481+0000","last_became_peered":"2023-06-23T18:03:41.789481+0000","last_unstale":"2023-06-23T18:03:42.922639+0000","last_undegraded":"2023-06-23T18:03:42.922639+0000","last_fullsized":"2023-06-23T18:03:42.922639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158754+0000","last_change":"2023-06-23T18:03:37.754662+0000","last_active":"2023-06-23T18:03:42.158754+0000","last_peered":"2023-06-23T18:03:42.158754+0000","last_clean":"2023-06-23T18:03:42.158754+0000","last_became_active":"2023-06-23T18:03:37.753982+0000","last_became_peered":"2023-06-23T18:03:37.753982+0000","last_unstale":"2023-06-23T18:03:42.158754+0000","last_undegraded":"2023-06-23T18:03:42.158754+0000","last_fullsized":"2023-06-23T18:03:42.158754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:40.534179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921128+0000","last_change":"2023-06-23T18:03:37.380222+0000","last_active":"2023-06-23T18:03:42.921128+0000","last_peered":"2023-06-23T18:03:42.921128+0000","last_clean":"2023-06-23T18:03:42.921128+0000","last_became_active":"2023-06-23T18:03:37.379866+0000","last_became_peered":"2023-06-23T18:03:37.379866+0000","last_unstale":"2023-06-23T18:03:42.921128+0000","last_undegraded":"2023-06-23T18:03:42.921128+0000","last_fullsized":"2023-06-23T18:03:42.921128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:12.333579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917855+0000","last_change":"2023-06-23T18:03:41.782310+0000","last_active":"2023-06-23T18:03:42.917855+0000","last_peered":"2023-06-23T18:03:42.917855+0000","last_clean":"2023-06-23T18:03:42.917855+0000","last_became_active":"2023-06-23T18:03:41.781820+0000","last_became_peered":"2023-06-23T18:03:41.781820+0000","last_unstale":"2023-06-23T18:03:42.917855+0000","last_undegraded":"2023-06-23T18:03:42.917855+0000","last_fullsized":"2023-06-23T18:03:42.917855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:39.845448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158787+0000","last_change":"2023-06-23T18:03:37.753742+0000","last_active":"2023-06-23T18:03:42.158787+0000","last_peered":"2023-06-23T18:03:42.158787+0000","last_clean":"2023-06-23T18:03:42.158787+0000","last_became_active":"2023-06-23T18:03:37.753391+0000","last_became_peered":"2023-06-23T18:03:37.753391+0000","last_unstale":"2023-06-23T18:03:42.158787+0000","last_undegraded":"2023-06-23T18:03:42.158787+0000","last_fullsized":"2023-06-23T18:03:42.158787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:54.787520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918436+0000","last_change":"2023-06-23T18:03:37.378501+0000","last_active":"2023-06-23T18:03:42.918436+0000","last_peered":"2023-06-23T18:03:42.918436+0000","last_clean":"2023-06-23T18:03:42.918436+0000","last_became_active":"2023-06-23T18:03:37.378222+0000","last_became_peered":"2023-06-23T18:03:37.378222+0000","last_unstale":"2023-06-23T18:03:42.918436+0000","last_undegraded":"2023-06-23T18:03:42.918436+0000","last_fullsized":"2023-06-23T18:03:42.918436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:49.965045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152626+0000","last_change":"2023-06-23T18:03:41.788999+0000","last_active":"2023-06-23T18:03:42.152626+0000","last_peered":"2023-06-23T18:03:42.152626+0000","last_clean":"2023-06-23T18:03:42.152626+0000","last_became_active":"2023-06-23T18:03:41.788606+0000","last_became_peered":"2023-06-23T18:03:41.788606+0000","last_unstale":"2023-06-23T18:03:42.152626+0000","last_undegraded":"2023-06-23T18:03:42.152626+0000","last_fullsized":"2023-06-23T18:03:42.152626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:40:16.265229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920805+0000","last_change":"2023-06-23T18:03:37.751267+0000","last_active":"2023-06-23T18:03:42.920805+0000","last_peered":"2023-06-23T18:03:42.920805+0000","last_clean":"2023-06-23T18:03:42.920805+0000","last_became_active":"2023-06-23T18:03:37.746037+0000","last_became_peered":"2023-06-23T18:03:37.746037+0000","last_unstale":"2023-06-23T18:03:42.920805+0000","last_undegraded":"2023-06-23T18:03:42.920805+0000","last_fullsized":"2023-06-23T18:03:42.920805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922834+0000","last_change":"2023-06-23T18:03:37.369505+0000","last_active":"2023-06-23T18:03:42.922834+0000","last_peered":"2023-06-23T18:03:42.922834+0000","last_clean":"2023-06-23T18:03:42.922834+0000","last_became_active":"2023-06-23T18:03:37.369174+0000","last_became_peered":"2023-06-23T18:03:37.369174+0000","last_unstale":"2023-06-23T18:03:42.922834+0000","last_undegraded":"2023-06-23T18:03:42.922834+0000","last_fullsized":"2023-06-23T18:03:42.922834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151984+0000","last_change":"2023-06-23T18:03:41.781321+0000","last_active":"2023-06-23T18:03:42.151984+0000","last_peered":"2023-06-23T18:03:42.151984+0000","last_clean":"2023-06-23T18:03:42.151984+0000","last_became_active":"2023-06-23T18:03:41.780831+0000","last_became_peered":"2023-06-23T18:03:41.780831+0000","last_unstale":"2023-06-23T18:03:42.151984+0000","last_undegraded":"2023-06-23T18:03:42.151984+0000","last_fullsized":"2023-06-23T18:03:42.151984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:53.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920510+0000","last_change":"2023-06-23T18:03:37.761108+0000","last_active":"2023-06-23T18:03:42.920510+0000","last_peered":"2023-06-23T18:03:42.920510+0000","last_clean":"2023-06-23T18:03:42.920510+0000","last_became_active":"2023-06-23T18:03:37.760041+0000","last_became_peered":"2023-06-23T18:03:37.760041+0000","last_unstale":"2023-06-23T18:03:42.920510+0000","last_undegraded":"2023-06-23T18:03:42.920510+0000","last_fullsized":"2023-06-23T18:03:42.920510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:46:38.396933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922050+0000","last_change":"2023-06-23T18:03:37.377838+0000","last_active":"2023-06-23T18:03:42.922050+0000","last_peered":"2023-06-23T18:03:42.922050+0000","last_clean":"2023-06-23T18:03:42.922050+0000","last_became_active":"2023-06-23T18:03:37.377381+0000","last_became_peered":"2023-06-23T18:03:37.377381+0000","last_unstale":"2023-06-23T18:03:42.922050+0000","last_undegraded":"2023-06-23T18:03:42.922050+0000","last_fullsized":"2023-06-23T18:03:42.922050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152297+0000","last_change":"2023-06-23T18:03:41.787926+0000","last_active":"2023-06-23T18:03:42.152297+0000","last_peered":"2023-06-23T18:03:42.152297+0000","last_clean":"2023-06-23T18:03:42.152297+0000","last_became_active":"2023-06-23T18:03:41.787539+0000","last_became_peered":"2023-06-23T18:03:41.787539+0000","last_unstale":"2023-06-23T18:03:42.152297+0000","last_undegraded":"2023-06-23T18:03:42.152297+0000","last_fullsized":"2023-06-23T18:03:42.152297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:49.966258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156824+0000","last_change":"2023-06-23T18:03:37.745296+0000","last_active":"2023-06-23T18:03:42.156824+0000","last_peered":"2023-06-23T18:03:42.156824+0000","last_clean":"2023-06-23T18:03:42.156824+0000","last_became_active":"2023-06-23T18:03:37.744996+0000","last_became_peered":"2023-06-23T18:03:37.744996+0000","last_unstale":"2023-06-23T18:03:42.156824+0000","last_undegraded":"2023-06-23T18:03:42.156824+0000","last_fullsized":"2023-06-23T18:03:42.156824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:07:37.054289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922271+0000","last_change":"2023-06-23T18:03:37.381405+0000","last_active":"2023-06-23T18:03:42.922271+0000","last_peered":"2023-06-23T18:03:42.922271+0000","last_clean":"2023-06-23T18:03:42.922271+0000","last_became_active":"2023-06-23T18:03:37.381038+0000","last_became_peered":"2023-06-23T18:03:37.381038+0000","last_unstale":"2023-06-23T18:03:42.922271+0000","last_undegraded":"2023-06-23T18:03:42.922271+0000","last_fullsized":"2023-06-23T18:03:42.922271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151761+0000","last_change":"2023-06-23T18:03:41.787211+0000","last_active":"2023-06-23T18:03:42.151761+0000","last_peered":"2023-06-23T18:03:42.151761+0000","last_clean":"2023-06-23T18:03:42.151761+0000","last_became_active":"2023-06-23T18:03:41.786809+0000","last_became_peered":"2023-06-23T18:03:41.786809+0000","last_unstale":"2023-06-23T18:03:42.151761+0000","last_undegraded":"2023-06-23T18:03:42.151761+0000","last_fullsized":"2023-06-23T18:03:42.151761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:42.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919723+0000","last_change":"2023-06-23T18:03:37.770649+0000","last_active":"2023-06-23T18:03:42.919723+0000","last_peered":"2023-06-23T18:03:42.919723+0000","last_clean":"2023-06-23T18:03:42.919723+0000","last_became_active":"2023-06-23T18:03:37.770326+0000","last_became_peered":"2023-06-23T18:03:37.770326+0000","last_unstale":"2023-06-23T18:03:42.919723+0000","last_undegraded":"2023-06-23T18:03:42.919723+0000","last_fullsized":"2023-06-23T18:03:42.919723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923872+0000","last_change":"2023-06-23T18:03:37.381617+0000","last_active":"2023-06-23T18:03:42.923872+0000","last_peered":"2023-06-23T18:03:42.923872+0000","last_clean":"2023-06-23T18:03:42.923872+0000","last_became_active":"2023-06-23T18:03:37.381342+0000","last_became_peered":"2023-06-23T18:03:37.381342+0000","last_unstale":"2023-06-23T18:03:42.923872+0000","last_undegraded":"2023-06-23T18:03:42.923872+0000","last_fullsized":"2023-06-23T18:03:42.923872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:31.692075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920724+0000","last_change":"2023-06-23T18:03:41.791679+0000","last_active":"2023-06-23T18:03:42.920724+0000","last_peered":"2023-06-23T18:03:42.920724+0000","last_clean":"2023-06-23T18:03:42.920724+0000","last_became_active":"2023-06-23T18:03:41.788101+0000","last_became_peered":"2023-06-23T18:03:41.788101+0000","last_unstale":"2023-06-23T18:03:42.920724+0000","last_undegraded":"2023-06-23T18:03:42.920724+0000","last_fullsized":"2023-06-23T18:03:42.920724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924128+0000","last_change":"2023-06-23T18:03:41.789672+0000","last_active":"2023-06-23T18:03:42.924128+0000","last_peered":"2023-06-23T18:03:42.924128+0000","last_clean":"2023-06-23T18:03:42.924128+0000","last_became_active":"2023-06-23T18:03:41.789258+0000","last_became_peered":"2023-06-23T18:03:41.789258+0000","last_unstale":"2023-06-23T18:03:42.924128+0000","last_undegraded":"2023-06-23T18:03:42.924128+0000","last_fullsized":"2023-06-23T18:03:42.924128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:35:03.864692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919206+0000","last_change":"2023-06-23T18:03:37.380523+0000","last_active":"2023-06-23T18:03:42.919206+0000","last_peered":"2023-06-23T18:03:42.919206+0000","last_clean":"2023-06-23T18:03:42.919206+0000","last_became_active":"2023-06-23T18:03:37.380210+0000","last_became_peered":"2023-06-23T18:03:37.380210+0000","last_unstale":"2023-06-23T18:03:42.919206+0000","last_undegraded":"2023-06-23T18:03:42.919206+0000","last_fullsized":"2023-06-23T18:03:42.919206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:57.050399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921568+0000","last_change":"2023-06-23T18:03:37.769174+0000","last_active":"2023-06-23T18:03:42.921568+0000","last_peered":"2023-06-23T18:03:42.921568+0000","last_clean":"2023-06-23T18:03:42.921568+0000","last_became_active":"2023-06-23T18:03:37.768922+0000","last_became_peered":"2023-06-23T18:03:37.768922+0000","last_unstale":"2023-06-23T18:03:42.921568+0000","last_undegraded":"2023-06-23T18:03:42.921568+0000","last_fullsized":"2023-06-23T18:03:42.921568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921045+0000","last_change":"2023-06-23T18:03:41.785038+0000","last_active":"2023-06-23T18:03:42.921045+0000","last_peered":"2023-06-23T18:03:42.921045+0000","last_clean":"2023-06-23T18:03:42.921045+0000","last_became_active":"2023-06-23T18:03:41.784124+0000","last_became_peered":"2023-06-23T18:03:41.784124+0000","last_unstale":"2023-06-23T18:03:42.921045+0000","last_undegraded":"2023-06-23T18:03:42.921045+0000","last_fullsized":"2023-06-23T18:03:42.921045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921205+0000","last_change":"2023-06-23T18:03:37.381450+0000","last_active":"2023-06-23T18:03:42.921205+0000","last_peered":"2023-06-23T18:03:42.921205+0000","last_clean":"2023-06-23T18:03:42.921205+0000","last_became_active":"2023-06-23T18:03:37.381137+0000","last_became_peered":"2023-06-23T18:03:37.381137+0000","last_unstale":"2023-06-23T18:03:42.921205+0000","last_undegraded":"2023-06-23T18:03:42.921205+0000","last_fullsized":"2023-06-23T18:03:42.921205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:04.690816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922338+0000","last_change":"2023-06-23T18:03:37.752538+0000","last_active":"2023-06-23T18:03:42.922338+0000","last_peered":"2023-06-23T18:03:42.922338+0000","last_clean":"2023-06-23T18:03:42.922338+0000","last_became_active":"2023-06-23T18:03:37.752276+0000","last_became_peered":"2023-06-23T18:03:37.752276+0000","last_unstale":"2023-06-23T18:03:42.922338+0000","last_undegraded":"2023-06-23T18:03:42.922338+0000","last_fullsized":"2023-06-23T18:03:42.922338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:23.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921015+0000","last_change":"2023-06-23T18:03:37.376067+0000","last_active":"2023-06-23T18:03:42.921015+0000","last_peered":"2023-06-23T18:03:42.921015+0000","last_clean":"2023-06-23T18:03:42.921015+0000","last_became_active":"2023-06-23T18:03:37.375582+0000","last_became_peered":"2023-06-23T18:03:37.375582+0000","last_unstale":"2023-06-23T18:03:42.921015+0000","last_undegraded":"2023-06-23T18:03:42.921015+0000","last_fullsized":"2023-06-23T18:03:42.921015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924004+0000","last_change":"2023-06-23T18:03:37.770794+0000","last_active":"2023-06-23T18:03:42.924004+0000","last_peered":"2023-06-23T18:03:42.924004+0000","last_clean":"2023-06-23T18:03:42.924004+0000","last_became_active":"2023-06-23T18:03:37.770508+0000","last_became_peered":"2023-06-23T18:03:37.770508+0000","last_unstale":"2023-06-23T18:03:42.924004+0000","last_undegraded":"2023-06-23T18:03:42.924004+0000","last_fullsized":"2023-06-23T18:03:42.924004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:27.966872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919085+0000","last_change":"2023-06-23T18:03:41.784964+0000","last_active":"2023-06-23T18:03:42.919085+0000","last_peered":"2023-06-23T18:03:42.919085+0000","last_clean":"2023-06-23T18:03:42.919085+0000","last_became_active":"2023-06-23T18:03:41.783875+0000","last_became_peered":"2023-06-23T18:03:41.783875+0000","last_unstale":"2023-06-23T18:03:42.919085+0000","last_undegraded":"2023-06-23T18:03:42.919085+0000","last_fullsized":"2023-06-23T18:03:42.919085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:03.189212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921280+0000","last_change":"2023-06-23T18:03:37.381471+0000","last_active":"2023-06-23T18:03:42.921280+0000","last_peered":"2023-06-23T18:03:42.921280+0000","last_clean":"2023-06-23T18:03:42.921280+0000","last_became_active":"2023-06-23T18:03:37.381080+0000","last_became_peered":"2023-06-23T18:03:37.381080+0000","last_unstale":"2023-06-23T18:03:42.921280+0000","last_undegraded":"2023-06-23T18:03:42.921280+0000","last_fullsized":"2023-06-23T18:03:42.921280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:36.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919682+0000","last_change":"2023-06-23T18:03:37.770191+0000","last_active":"2023-06-23T18:03:42.919682+0000","last_peered":"2023-06-23T18:03:42.919682+0000","last_clean":"2023-06-23T18:03:42.919682+0000","last_became_active":"2023-06-23T18:03:37.769918+0000","last_became_peered":"2023-06-23T18:03:37.769918+0000","last_unstale":"2023-06-23T18:03:42.919682+0000","last_undegraded":"2023-06-23T18:03:42.919682+0000","last_fullsized":"2023-06-23T18:03:42.919682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:36.439982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924483+0000","last_change":"2023-06-23T18:03:41.789295+0000","last_active":"2023-06-23T18:03:42.924483+0000","last_peered":"2023-06-23T18:03:42.924483+0000","last_clean":"2023-06-23T18:03:42.924483+0000","last_became_active":"2023-06-23T18:03:41.788890+0000","last_became_peered":"2023-06-23T18:03:41.788890+0000","last_unstale":"2023-06-23T18:03:42.924483+0000","last_undegraded":"2023-06-23T18:03:42.924483+0000","last_fullsized":"2023-06-23T18:03:42.924483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:36.106369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918865+0000","last_change":"2023-06-23T18:03:37.759952+0000","last_active":"2023-06-23T18:03:42.918865+0000","last_peered":"2023-06-23T18:03:42.918865+0000","last_clean":"2023-06-23T18:03:42.918865+0000","last_became_active":"2023-06-23T18:03:37.759685+0000","last_became_peered":"2023-06-23T18:03:37.759685+0000","last_unstale":"2023-06-23T18:03:42.918865+0000","last_undegraded":"2023-06-23T18:03:42.918865+0000","last_fullsized":"2023-06-23T18:03:42.918865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:01:11.356694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916611+0000","last_change":"2023-06-23T18:03:37.761169+0000","last_active":"2023-06-23T18:03:42.916611+0000","last_peered":"2023-06-23T18:03:42.916611+0000","last_clean":"2023-06-23T18:03:42.916611+0000","last_became_active":"2023-06-23T18:03:37.760915+0000","last_became_peered":"2023-06-23T18:03:37.760915+0000","last_unstale":"2023-06-23T18:03:42.916611+0000","last_undegraded":"2023-06-23T18:03:42.916611+0000","last_fullsized":"2023-06-23T18:03:42.916611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:15:41.693241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157660+0000","last_change":"2023-06-23T18:03:37.756608+0000","last_active":"2023-06-23T18:03:42.157660+0000","last_peered":"2023-06-23T18:03:42.157660+0000","last_clean":"2023-06-23T18:03:42.157660+0000","last_became_active":"2023-06-23T18:03:37.756267+0000","last_became_peered":"2023-06-23T18:03:37.756267+0000","last_unstale":"2023-06-23T18:03:42.157660+0000","last_undegraded":"2023-06-23T18:03:42.157660+0000","last_fullsized":"2023-06-23T18:03:42.157660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:39:56.896505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917984+0000","last_change":"2023-06-23T18:03:37.768762+0000","last_active":"2023-06-23T18:03:42.917984+0000","last_peered":"2023-06-23T18:03:42.917984+0000","last_clean":"2023-06-23T18:03:42.917984+0000","last_became_active":"2023-06-23T18:03:37.768493+0000","last_became_peered":"2023-06-23T18:03:37.768493+0000","last_unstale":"2023-06-23T18:03:42.917984+0000","last_undegraded":"2023-06-23T18:03:42.917984+0000","last_fullsized":"2023-06-23T18:03:42.917984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:47.275516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918340+0000","last_change":"2023-06-23T18:03:37.770023+0000","last_active":"2023-06-23T18:03:42.918340+0000","last_peered":"2023-06-23T18:03:42.918340+0000","last_clean":"2023-06-23T18:03:42.918340+0000","last_became_active":"2023-06-23T18:03:37.769776+0000","last_became_peered":"2023-06-23T18:03:37.769776+0000","last_unstale":"2023-06-23T18:03:42.918340+0000","last_undegraded":"2023-06-23T18:03:42.918340+0000","last_fullsized":"2023-06-23T18:03:42.918340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923845+0000","last_change":"2023-06-23T18:03:37.772405+0000","last_active":"2023-06-23T18:03:42.923845+0000","last_peered":"2023-06-23T18:03:42.923845+0000","last_clean":"2023-06-23T18:03:42.923845+0000","last_became_active":"2023-06-23T18:03:37.772086+0000","last_became_peered":"2023-06-23T18:03:37.772086+0000","last_unstale":"2023-06-23T18:03:42.923845+0000","last_undegraded":"2023-06-23T18:03:42.923845+0000","last_fullsized":"2023-06-23T18:03:42.923845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:49.758628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919917+0000","last_change":"2023-06-23T18:03:37.768948+0000","last_active":"2023-06-23T18:03:42.919917+0000","last_peered":"2023-06-23T18:03:42.919917+0000","last_clean":"2023-06-23T18:03:42.919917+0000","last_became_active":"2023-06-23T18:03:37.768355+0000","last_became_peered":"2023-06-23T18:03:37.768355+0000","last_unstale":"2023-06-23T18:03:42.919917+0000","last_undegraded":"2023-06-23T18:03:42.919917+0000","last_fullsized":"2023-06-23T18:03:42.919917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919527+0000","last_change":"2023-06-23T18:03:37.764486+0000","last_active":"2023-06-23T18:03:42.919527+0000","last_peered":"2023-06-23T18:03:42.919527+0000","last_clean":"2023-06-23T18:03:42.919527+0000","last_became_active":"2023-06-23T18:03:37.755419+0000","last_became_peered":"2023-06-23T18:03:37.755419+0000","last_unstale":"2023-06-23T18:03:42.919527+0000","last_undegraded":"2023-06-23T18:03:42.919527+0000","last_fullsized":"2023-06-23T18:03:42.919527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921001+0000","last_change":"2023-06-23T18:03:37.770648+0000","last_active":"2023-06-23T18:03:42.921001+0000","last_peered":"2023-06-23T18:03:42.921001+0000","last_clean":"2023-06-23T18:03:42.921001+0000","last_became_active":"2023-06-23T18:03:37.770271+0000","last_became_peered":"2023-06-23T18:03:37.770271+0000","last_unstale":"2023-06-23T18:03:42.921001+0000","last_undegraded":"2023-06-23T18:03:42.921001+0000","last_fullsized":"2023-06-23T18:03:42.921001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916514+0000","last_change":"2023-06-23T18:03:37.767363+0000","last_active":"2023-06-23T18:03:42.916514+0000","last_peered":"2023-06-23T18:03:42.916514+0000","last_clean":"2023-06-23T18:03:42.916514+0000","last_became_active":"2023-06-23T18:03:37.766918+0000","last_became_peered":"2023-06-23T18:03:37.766918+0000","last_unstale":"2023-06-23T18:03:42.916514+0000","last_undegraded":"2023-06-23T18:03:42.916514+0000","last_fullsized":"2023-06-23T18:03:42.916514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:31.121336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920725+0000","last_change":"2023-06-23T18:03:37.757526+0000","last_active":"2023-06-23T18:03:42.920725+0000","last_peered":"2023-06-23T18:03:42.920725+0000","last_clean":"2023-06-23T18:03:42.920725+0000","last_became_active":"2023-06-23T18:03:37.757223+0000","last_became_peered":"2023-06-23T18:03:37.757223+0000","last_unstale":"2023-06-23T18:03:42.920725+0000","last_undegraded":"2023-06-23T18:03:42.920725+0000","last_fullsized":"2023-06-23T18:03:42.920725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921071+0000","last_change":"2023-06-23T18:03:37.770932+0000","last_active":"2023-06-23T18:03:42.921071+0000","last_peered":"2023-06-23T18:03:42.921071+0000","last_clean":"2023-06-23T18:03:42.921071+0000","last_became_active":"2023-06-23T18:03:37.770633+0000","last_became_peered":"2023-06-23T18:03:37.770633+0000","last_unstale":"2023-06-23T18:03:42.921071+0000","last_undegraded":"2023-06-23T18:03:42.921071+0000","last_fullsized":"2023-06-23T18:03:42.921071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158738+0000","last_change":"2023-06-23T18:03:37.757510+0000","last_active":"2023-06-23T18:03:42.158738+0000","last_peered":"2023-06-23T18:03:42.158738+0000","last_clean":"2023-06-23T18:03:42.158738+0000","last_became_active":"2023-06-23T18:03:37.757203+0000","last_became_peered":"2023-06-23T18:03:37.757203+0000","last_unstale":"2023-06-23T18:03:42.158738+0000","last_undegraded":"2023-06-23T18:03:42.158738+0000","last_fullsized":"2023-06-23T18:03:42.158738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:25:37.629512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153090+0000","last_change":"2023-06-23T18:03:37.760931+0000","last_active":"2023-06-23T18:03:42.153090+0000","last_peered":"2023-06-23T18:03:42.153090+0000","last_clean":"2023-06-23T18:03:42.153090+0000","last_became_active":"2023-06-23T18:03:37.760672+0000","last_became_peered":"2023-06-23T18:03:37.760672+0000","last_unstale":"2023-06-23T18:03:42.153090+0000","last_undegraded":"2023-06-23T18:03:42.153090+0000","last_fullsized":"2023-06-23T18:03:42.153090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:35.350379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158558+0000","last_change":"2023-06-23T18:03:37.761315+0000","last_active":"2023-06-23T18:03:42.158558+0000","last_peered":"2023-06-23T18:03:42.158558+0000","last_clean":"2023-06-23T18:03:42.158558+0000","last_became_active":"2023-06-23T18:03:37.760975+0000","last_became_peered":"2023-06-23T18:03:37.760975+0000","last_unstale":"2023-06-23T18:03:42.158558+0000","last_undegraded":"2023-06-23T18:03:42.158558+0000","last_fullsized":"2023-06-23T18:03:42.158558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:04.033803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922192+0000","last_change":"2023-06-23T18:03:37.754659+0000","last_active":"2023-06-23T18:03:42.922192+0000","last_peered":"2023-06-23T18:03:42.922192+0000","last_clean":"2023-06-23T18:03:42.922192+0000","last_became_active":"2023-06-23T18:03:37.751277+0000","last_became_peered":"2023-06-23T18:03:37.751277+0000","last_unstale":"2023-06-23T18:03:42.922192+0000","last_undegraded":"2023-06-23T18:03:42.922192+0000","last_fullsized":"2023-06-23T18:03:42.922192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156785+0000","last_change":"2023-06-23T18:03:37.752494+0000","last_active":"2023-06-23T18:03:42.156785+0000","last_peered":"2023-06-23T18:03:42.156785+0000","last_clean":"2023-06-23T18:03:42.156785+0000","last_became_active":"2023-06-23T18:03:37.752149+0000","last_became_peered":"2023-06-23T18:03:37.752149+0000","last_unstale":"2023-06-23T18:03:42.156785+0000","last_undegraded":"2023-06-23T18:03:42.156785+0000","last_fullsized":"2023-06-23T18:03:42.156785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:37.654382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921056+0000","last_change":"2023-06-23T18:03:37.775997+0000","last_active":"2023-06-23T18:03:42.921056+0000","last_peered":"2023-06-23T18:03:42.921056+0000","last_clean":"2023-06-23T18:03:42.921056+0000","last_became_active":"2023-06-23T18:03:37.775685+0000","last_became_peered":"2023-06-23T18:03:37.775685+0000","last_unstale":"2023-06-23T18:03:42.921056+0000","last_undegraded":"2023-06-23T18:03:42.921056+0000","last_fullsized":"2023-06-23T18:03:42.921056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923426+0000","last_change":"2023-06-23T18:03:37.763367+0000","last_active":"2023-06-23T18:03:42.923426+0000","last_peered":"2023-06-23T18:03:42.923426+0000","last_clean":"2023-06-23T18:03:42.923426+0000","last_became_active":"2023-06-23T18:03:37.763035+0000","last_became_peered":"2023-06-23T18:03:37.763035+0000","last_unstale":"2023-06-23T18:03:42.923426+0000","last_undegraded":"2023-06-23T18:03:42.923426+0000","last_fullsized":"2023-06-23T18:03:42.923426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152175+0000","last_change":"2023-06-23T18:03:37.758554+0000","last_active":"2023-06-23T18:03:42.152175+0000","last_peered":"2023-06-23T18:03:42.152175+0000","last_clean":"2023-06-23T18:03:42.152175+0000","last_became_active":"2023-06-23T18:03:37.758307+0000","last_became_peered":"2023-06-23T18:03:37.758307+0000","last_unstale":"2023-06-23T18:03:42.152175+0000","last_undegraded":"2023-06-23T18:03:42.152175+0000","last_fullsized":"2023-06-23T18:03:42.152175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:23:09.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152731+0000","last_change":"2023-06-23T18:03:37.760705+0000","last_active":"2023-06-23T18:03:42.152731+0000","last_peered":"2023-06-23T18:03:42.152731+0000","last_clean":"2023-06-23T18:03:42.152731+0000","last_became_active":"2023-06-23T18:03:37.759846+0000","last_became_peered":"2023-06-23T18:03:37.759846+0000","last_unstale":"2023-06-23T18:03:42.152731+0000","last_undegraded":"2023-06-23T18:03:42.152731+0000","last_fullsized":"2023-06-23T18:03:42.152731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:26.801591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158644+0000","last_change":"2023-06-23T18:03:37.757927+0000","last_active":"2023-06-23T18:03:42.158644+0000","last_peered":"2023-06-23T18:03:42.158644+0000","last_clean":"2023-06-23T18:03:42.158644+0000","last_became_active":"2023-06-23T18:03:37.757635+0000","last_became_peered":"2023-06-23T18:03:37.757635+0000","last_unstale":"2023-06-23T18:03:42.158644+0000","last_undegraded":"2023-06-23T18:03:42.158644+0000","last_fullsized":"2023-06-23T18:03:42.158644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:26.427732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923450+0000","last_change":"2023-06-23T18:03:37.768282+0000","last_active":"2023-06-23T18:03:42.923450+0000","last_peered":"2023-06-23T18:03:42.923450+0000","last_clean":"2023-06-23T18:03:42.923450+0000","last_became_active":"2023-06-23T18:03:37.768038+0000","last_became_peered":"2023-06-23T18:03:37.768038+0000","last_unstale":"2023-06-23T18:03:42.923450+0000","last_undegraded":"2023-06-23T18:03:42.923450+0000","last_fullsized":"2023-06-23T18:03:42.923450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922018+0000","last_change":"2023-06-23T18:03:37.752981+0000","last_active":"2023-06-23T18:03:42.922018+0000","last_peered":"2023-06-23T18:03:42.922018+0000","last_clean":"2023-06-23T18:03:42.922018+0000","last_became_active":"2023-06-23T18:03:37.752291+0000","last_became_peered":"2023-06-23T18:03:37.752291+0000","last_unstale":"2023-06-23T18:03:42.922018+0000","last_undegraded":"2023-06-23T18:03:42.922018+0000","last_fullsized":"2023-06-23T18:03:42.922018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919440+0000","last_change":"2023-06-23T18:03:37.756558+0000","last_active":"2023-06-23T18:03:42.919440+0000","last_peered":"2023-06-23T18:03:42.919440+0000","last_clean":"2023-06-23T18:03:42.919440+0000","last_became_active":"2023-06-23T18:03:37.756044+0000","last_became_peered":"2023-06-23T18:03:37.756044+0000","last_unstale":"2023-06-23T18:03:42.919440+0000","last_undegraded":"2023-06-23T18:03:42.919440+0000","last_fullsized":"2023-06-23T18:03:42.919440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920901+0000","last_change":"2023-06-23T18:03:37.775992+0000","last_active":"2023-06-23T18:03:42.920901+0000","last_peered":"2023-06-23T18:03:42.920901+0000","last_clean":"2023-06-23T18:03:42.920901+0000","last_became_active":"2023-06-23T18:03:37.775667+0000","last_became_peered":"2023-06-23T18:03:37.775667+0000","last_unstale":"2023-06-23T18:03:42.920901+0000","last_undegraded":"2023-06-23T18:03:42.920901+0000","last_fullsized":"2023-06-23T18:03:42.920901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157021+0000","last_change":"2023-06-23T18:03:37.752811+0000","last_active":"2023-06-23T18:03:42.157021+0000","last_peered":"2023-06-23T18:03:42.157021+0000","last_clean":"2023-06-23T18:03:42.157021+0000","last_became_active":"2023-06-23T18:03:37.752518+0000","last_became_peered":"2023-06-23T18:03:37.752518+0000","last_unstale":"2023-06-23T18:03:42.157021+0000","last_undegraded":"2023-06-23T18:03:42.157021+0000","last_fullsized":"2023-06-23T18:03:42.157021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:26.870651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920914+0000","last_change":"2023-06-23T18:03:37.757769+0000","last_active":"2023-06-23T18:03:42.920914+0000","last_peered":"2023-06-23T18:03:42.920914+0000","last_clean":"2023-06-23T18:03:42.920914+0000","last_became_active":"2023-06-23T18:03:37.757520+0000","last_became_peered":"2023-06-23T18:03:37.757520+0000","last_unstale":"2023-06-23T18:03:42.920914+0000","last_undegraded":"2023-06-23T18:03:42.920914+0000","last_fullsized":"2023-06-23T18:03:42.920914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919630+0000","last_change":"2023-06-23T18:03:37.755865+0000","last_active":"2023-06-23T18:03:42.919630+0000","last_peered":"2023-06-23T18:03:42.919630+0000","last_clean":"2023-06-23T18:03:42.919630+0000","last_became_active":"2023-06-23T18:03:37.755153+0000","last_became_peered":"2023-06-23T18:03:37.755153+0000","last_unstale":"2023-06-23T18:03:42.919630+0000","last_undegraded":"2023-06-23T18:03:42.919630+0000","last_fullsized":"2023-06-23T18:03:42.919630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:42:12.603650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158475+0000","last_change":"2023-06-23T18:03:37.754864+0000","last_active":"2023-06-23T18:03:42.158475+0000","last_peered":"2023-06-23T18:03:42.158475+0000","last_clean":"2023-06-23T18:03:42.158475+0000","last_became_active":"2023-06-23T18:03:37.754386+0000","last_became_peered":"2023-06-23T18:03:37.754386+0000","last_unstale":"2023-06-23T18:03:42.158475+0000","last_undegraded":"2023-06-23T18:03:42.158475+0000","last_fullsized":"2023-06-23T18:03:42.158475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:30:06.917401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922699+0000","last_change":"2023-06-23T18:03:37.757620+0000","last_active":"2023-06-23T18:03:42.922699+0000","last_peered":"2023-06-23T18:03:42.922699+0000","last_clean":"2023-06-23T18:03:42.922699+0000","last_became_active":"2023-06-23T18:03:37.757346+0000","last_became_peered":"2023-06-23T18:03:37.757346+0000","last_unstale":"2023-06-23T18:03:42.922699+0000","last_undegraded":"2023-06-23T18:03:42.922699+0000","last_fullsized":"2023-06-23T18:03:42.922699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:59.832808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152893+0000","last_change":"2023-06-23T18:03:37.759638+0000","last_active":"2023-06-23T18:03:42.152893+0000","last_peered":"2023-06-23T18:03:42.152893+0000","last_clean":"2023-06-23T18:03:42.152893+0000","last_became_active":"2023-06-23T18:03:37.759380+0000","last_became_peered":"2023-06-23T18:03:37.759380+0000","last_unstale":"2023-06-23T18:03:42.152893+0000","last_undegraded":"2023-06-23T18:03:42.152893+0000","last_fullsized":"2023-06-23T18:03:42.152893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:44.739308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31300,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340540,"statfs":{"total":96636764160,"available":96604712960,"internally_reserved":0,"allocated":258048,"data_stored":104812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":851968,"data_stored":695672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:04:04.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:04:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 Processor -- start 2023-06-23T18:04:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 -- start start 2023-06-23T18:04:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 0x7f3e140afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3e140affd0 0x7f3e140c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 0x7f3e140cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e141590a0 con 0x7f3e140affd0 2023-06-23T18:04:04.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140b0420 con 0x7f3e140c93a0 2023-06-23T18:04:04.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e198d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140c9b50 con 0x7f3e14070280 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3e140affd0 0x7f3e140c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3e140affd0 0x7f3e140c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44158/0 (socket says 172.21.15.22:44158) 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 -- 172.21.15.22:0/988424055 learned_addr learned my addr 172.21.15.22:0/988424055 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 -- 172.21.15.22:0/988424055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 msgr2=0x7f3e140afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e190d8700 1 --2- 172.21.15.22:0/988424055 >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 0x7f3e140cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e188d7700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 0x7f3e140afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 0x7f3e140afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 -- 172.21.15.22:0/988424055 >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 msgr2=0x7f3e140cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 --2- 172.21.15.22:0/988424055 >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 0x7f3e140cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 -- 172.21.15.22:0/988424055 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140c9e30 con 0x7f3e140affd0 2023-06-23T18:04:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e190d8700 1 --2- 172.21.15.22:0/988424055 >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 0x7f3e140cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e188d7700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 0x7f3e140afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e13fff700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e140affd0 0x7f3e140c8e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e1406d0b0 tx=0x7f3dfc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b57fa9d6caf74ea server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e137fe700 1 -- 172.21.15.22:0/988424055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00f8a0 con 0x7f3e140affd0 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e137fe700 1 -- 172.21.15.22:0/988424055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00fa00 con 0x7f3e140affd0 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.511+0000 7f3e137fe700 1 -- 172.21.15.22:0/988424055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00fd90 con 0x7f3e140affd0 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- 172.21.15.22:0/988424055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e140affd0 msgr2=0x7f3e140c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e140affd0 0x7f3e140c8e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e1406d0b0 tx=0x7f3dfc009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- 172.21.15.22:0/988424055 shutdown_connections 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 --2- 172.21.15.22:0/988424055 >> v2:172.21.15.169:3300/0 conn(0x7f3e140c93a0 0x7f3e140cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e140affd0 0x7f3e140c8e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 --2- 172.21.15.22:0/988424055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e14070280 0x7f3e140afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- 172.21.15.22:0/988424055 >> 172.21.15.22:0/988424055 conn(0x7f3e1406a5f0 msgr2=0x7f3e140caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- 172.21.15.22:0/988424055 shutdown_connections 2023-06-23T18:04:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- 172.21.15.22:0/988424055 wait complete. 2023-06-23T18:04:04.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 Processor -- start 2023-06-23T18:04:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.515+0000 7f3e198d9700 1 -- start start 2023-06-23T18:04:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e140affd0 0x7f3e140ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e140c93a0 0x7f3e141f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14159760 con 0x7f3e14070280 2023-06-23T18:04:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141f2210 con 0x7f3e140affd0 2023-06-23T18:04:04.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141f24f0 con 0x7f3e140c93a0 2023-06-23T18:04:04.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44172/0 (socket says 172.21.15.22:44172) 2023-06-23T18:04:04.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 -- 172.21.15.22:0/1547749566 learned_addr learned my addr 172.21.15.22:0/1547749566 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:04.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e190d8700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e140c93a0 0x7f3e141f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e13fff700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e140affd0 0x7f3e140ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 -- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e140c93a0 msgr2=0x7f3e141f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e140c93a0 0x7f3e141f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 -- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e140affd0 msgr2=0x7f3e140ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e140affd0 0x7f3e140ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 -- 172.21.15.22:0/1547749566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009570 con 0x7f3e14070280 2023-06-23T18:04:04.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e188d7700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c770 tx=0x7f3e0400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04011800 con 0x7f3e14070280 2023-06-23T18:04:04.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04011960 con 0x7f3e14070280 2023-06-23T18:04:04.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e14159380 con 0x7f3e14070280 2023-06-23T18:04:04.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04011d10 con 0x7f3e14070280 2023-06-23T18:04:04.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140aba90 con 0x7f3e14070280 2023-06-23T18:04:04.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.519+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e141599d0 con 0x7f3e14070280 2023-06-23T18:04:04.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.523+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3e04018360 con 0x7f3e14070280 2023-06-23T18:04:04.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.523+0000 7f3e117fa700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 0x7f3df80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.523+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e04083ea0 con 0x7f3e14070280 2023-06-23T18:04:04.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.523+0000 7f3e13fff700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 0x7f3df80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.523+0000 7f3e13fff700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 0x7f3df80687a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc01b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.531+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3e040510c0 con 0x7f3e14070280 2023-06-23T18:04:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.683+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e1415a2a0 con 0x7f3df80662f0 2023-06-23T18:04:04.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.699+0000 7f3e117fa700 1 -- 172.21.15.22:0/1547749566 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367249 (secure 0 0 0) 0x7f3dfc00fe50 con 0x7f3df80662f0 2023-06-23T18:04:04.718 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 msgr2=0x7f3df80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 0x7f3df80687a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc01b040 comp rx=0 tx=0).stop 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 msgr2=0x7f3e14159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c770 tx=0x7f3e0400ca80 comp rx=0 tx=0).stop 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 shutdown_connections 2023-06-23T18:04:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e140c93a0 0x7f3e141f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df80662f0 0x7f3df80687a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e140affd0 0x7f3e140ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 --2- 172.21.15.22:0/1547749566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e14070280 0x7f3e14159b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 >> 172.21.15.22:0/1547749566 conn(0x7f3e1406a5f0 msgr2=0x7f3e140af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 shutdown_connections 2023-06-23T18:04:04.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.703+0000 7f3e198d9700 1 -- 172.21.15.22:0/1547749566 wait complete. 2023-06-23T18:04:04.722 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:04:04.732 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-06-23T18:04:03.615910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594959,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235208,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":3284992,"data_stored":2027694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004683"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915552+0000","last_change":"2023-06-23T18:03:41.789565+0000","last_active":"2023-06-23T18:03:42.915552+0000","last_peered":"2023-06-23T18:03:42.915552+0000","last_clean":"2023-06-23T18:03:42.915552+0000","last_became_active":"2023-06-23T18:03:41.787653+0000","last_became_peered":"2023-06-23T18:03:41.787653+0000","last_unstale":"2023-06-23T18:03:42.915552+0000","last_undegraded":"2023-06-23T18:03:42.915552+0000","last_fullsized":"2023-06-23T18:03:42.915552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922374+0000","last_change":"2023-06-23T18:03:37.377461+0000","last_active":"2023-06-23T18:03:42.922374+0000","last_peered":"2023-06-23T18:03:42.922374+0000","last_clean":"2023-06-23T18:03:42.922374+0000","last_became_active":"2023-06-23T18:03:37.377017+0000","last_became_peered":"2023-06-23T18:03:37.377017+0000","last_unstale":"2023-06-23T18:03:42.922374+0000","last_undegraded":"2023-06-23T18:03:42.922374+0000","last_fullsized":"2023-06-23T18:03:42.922374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157012+0000","last_change":"2023-06-23T18:03:37.756400+0000","last_active":"2023-06-23T18:03:42.157012+0000","last_peered":"2023-06-23T18:03:42.157012+0000","last_clean":"2023-06-23T18:03:42.157012+0000","last_became_active":"2023-06-23T18:03:37.755870+0000","last_became_peered":"2023-06-23T18:03:37.755870+0000","last_unstale":"2023-06-23T18:03:42.157012+0000","last_undegraded":"2023-06-23T18:03:42.157012+0000","last_fullsized":"2023-06-23T18:03:42.157012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:35:48.712508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157286+0000","last_change":"2023-06-23T18:03:41.784738+0000","last_active":"2023-06-23T18:03:42.157286+0000","last_peered":"2023-06-23T18:03:42.157286+0000","last_clean":"2023-06-23T18:03:42.157286+0000","last_became_active":"2023-06-23T18:03:41.784348+0000","last_became_peered":"2023-06-23T18:03:41.784348+0000","last_unstale":"2023-06-23T18:03:42.157286+0000","last_undegraded":"2023-06-23T18:03:42.157286+0000","last_fullsized":"2023-06-23T18:03:42.157286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:06:40.211756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920797+0000","last_change":"2023-06-23T18:03:37.364428+0000","last_active":"2023-06-23T18:03:42.920797+0000","last_peered":"2023-06-23T18:03:42.920797+0000","last_clean":"2023-06-23T18:03:42.920797+0000","last_became_active":"2023-06-23T18:03:37.364114+0000","last_became_peered":"2023-06-23T18:03:37.364114+0000","last_unstale":"2023-06-23T18:03:42.920797+0000","last_undegraded":"2023-06-23T18:03:42.920797+0000","last_fullsized":"2023-06-23T18:03:42.920797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:53.340210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917007+0000","last_change":"2023-06-23T18:03:37.761252+0000","last_active":"2023-06-23T18:03:42.917007+0000","last_peered":"2023-06-23T18:03:42.917007+0000","last_clean":"2023-06-23T18:03:42.917007+0000","last_became_active":"2023-06-23T18:03:37.760991+0000","last_became_peered":"2023-06-23T18:03:37.760991+0000","last_unstale":"2023-06-23T18:03:42.917007+0000","last_undegraded":"2023-06-23T18:03:42.917007+0000","last_fullsized":"2023-06-23T18:03:42.917007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:14:26.556145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922896+0000","last_change":"2023-06-23T18:03:41.789561+0000","last_active":"2023-06-23T18:03:42.922896+0000","last_peered":"2023-06-23T18:03:42.922896+0000","last_clean":"2023-06-23T18:03:42.922896+0000","last_became_active":"2023-06-23T18:03:41.788768+0000","last_became_peered":"2023-06-23T18:03:41.788768+0000","last_unstale":"2023-06-23T18:03:42.922896+0000","last_undegraded":"2023-06-23T18:03:42.922896+0000","last_fullsized":"2023-06-23T18:03:42.922896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924155+0000","last_change":"2023-06-23T18:03:37.377124+0000","last_active":"2023-06-23T18:03:42.924155+0000","last_peered":"2023-06-23T18:03:42.924155+0000","last_clean":"2023-06-23T18:03:42.924155+0000","last_became_active":"2023-06-23T18:03:37.376829+0000","last_became_peered":"2023-06-23T18:03:37.376829+0000","last_unstale":"2023-06-23T18:03:42.924155+0000","last_undegraded":"2023-06-23T18:03:42.924155+0000","last_fullsized":"2023-06-23T18:03:42.924155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157618+0000","last_change":"2023-06-23T18:03:37.754592+0000","last_active":"2023-06-23T18:03:42.157618+0000","last_peered":"2023-06-23T18:03:42.157618+0000","last_clean":"2023-06-23T18:03:42.157618+0000","last_became_active":"2023-06-23T18:03:37.754279+0000","last_became_peered":"2023-06-23T18:03:37.754279+0000","last_unstale":"2023-06-23T18:03:42.157618+0000","last_undegraded":"2023-06-23T18:03:42.157618+0000","last_fullsized":"2023-06-23T18:03:42.157618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:05:29.058510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920916+0000","last_change":"2023-06-23T18:03:41.787327+0000","last_active":"2023-06-23T18:03:42.920916+0000","last_peered":"2023-06-23T18:03:42.920916+0000","last_clean":"2023-06-23T18:03:42.920916+0000","last_became_active":"2023-06-23T18:03:41.786664+0000","last_became_peered":"2023-06-23T18:03:41.786664+0000","last_unstale":"2023-06-23T18:03:42.920916+0000","last_undegraded":"2023-06-23T18:03:42.920916+0000","last_fullsized":"2023-06-23T18:03:42.920916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922619+0000","last_change":"2023-06-23T18:03:37.377468+0000","last_active":"2023-06-23T18:03:42.922619+0000","last_peered":"2023-06-23T18:03:42.922619+0000","last_clean":"2023-06-23T18:03:42.922619+0000","last_became_active":"2023-06-23T18:03:37.376653+0000","last_became_peered":"2023-06-23T18:03:37.376653+0000","last_unstale":"2023-06-23T18:03:42.922619+0000","last_undegraded":"2023-06-23T18:03:42.922619+0000","last_fullsized":"2023-06-23T18:03:42.922619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157926+0000","last_change":"2023-06-23T18:03:37.761926+0000","last_active":"2023-06-23T18:03:42.157926+0000","last_peered":"2023-06-23T18:03:42.157926+0000","last_clean":"2023-06-23T18:03:42.157926+0000","last_became_active":"2023-06-23T18:03:37.761621+0000","last_became_peered":"2023-06-23T18:03:37.761621+0000","last_unstale":"2023-06-23T18:03:42.157926+0000","last_undegraded":"2023-06-23T18:03:42.157926+0000","last_fullsized":"2023-06-23T18:03:42.157926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:21:20.322939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918838+0000","last_change":"2023-06-23T18:03:41.782917+0000","last_active":"2023-06-23T18:03:42.918838+0000","last_peered":"2023-06-23T18:03:42.918838+0000","last_clean":"2023-06-23T18:03:42.918838+0000","last_became_active":"2023-06-23T18:03:41.782516+0000","last_became_peered":"2023-06-23T18:03:41.782516+0000","last_unstale":"2023-06-23T18:03:42.918838+0000","last_undegraded":"2023-06-23T18:03:42.918838+0000","last_fullsized":"2023-06-23T18:03:42.918838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:32:03.238274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920996+0000","last_change":"2023-06-23T18:03:37.379511+0000","last_active":"2023-06-23T18:03:42.920996+0000","last_peered":"2023-06-23T18:03:42.920996+0000","last_clean":"2023-06-23T18:03:42.920996+0000","last_became_active":"2023-06-23T18:03:37.379252+0000","last_became_peered":"2023-06-23T18:03:37.379252+0000","last_unstale":"2023-06-23T18:03:42.920996+0000","last_undegraded":"2023-06-23T18:03:42.920996+0000","last_fullsized":"2023-06-23T18:03:42.920996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:58.823980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153021+0000","last_change":"2023-06-23T18:03:37.768862+0000","last_active":"2023-06-23T18:03:42.153021+0000","last_peered":"2023-06-23T18:03:42.153021+0000","last_clean":"2023-06-23T18:03:42.153021+0000","last_became_active":"2023-06-23T18:03:37.768610+0000","last_became_peered":"2023-06-23T18:03:37.768610+0000","last_unstale":"2023-06-23T18:03:42.153021+0000","last_undegraded":"2023-06-23T18:03:42.153021+0000","last_fullsized":"2023-06-23T18:03:42.153021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:10:03.453386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158181+0000","last_change":"2023-06-23T18:03:41.781852+0000","last_active":"2023-06-23T18:03:42.158181+0000","last_peered":"2023-06-23T18:03:42.158181+0000","last_clean":"2023-06-23T18:03:42.158181+0000","last_became_active":"2023-06-23T18:03:41.781321+0000","last_became_peered":"2023-06-23T18:03:41.781321+0000","last_unstale":"2023-06-23T18:03:42.158181+0000","last_undegraded":"2023-06-23T18:03:42.158181+0000","last_fullsized":"2023-06-23T18:03:42.158181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:21:44.946317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157942+0000","last_change":"2023-06-23T18:03:37.372568+0000","last_active":"2023-06-23T18:03:42.157942+0000","last_peered":"2023-06-23T18:03:42.157942+0000","last_clean":"2023-06-23T18:03:42.157942+0000","last_became_active":"2023-06-23T18:03:37.372256+0000","last_became_peered":"2023-06-23T18:03:37.372256+0000","last_unstale":"2023-06-23T18:03:42.157942+0000","last_undegraded":"2023-06-23T18:03:42.157942+0000","last_fullsized":"2023-06-23T18:03:42.157942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:01:27.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919391+0000","last_change":"2023-06-23T18:03:37.761387+0000","last_active":"2023-06-23T18:03:42.919391+0000","last_peered":"2023-06-23T18:03:42.919391+0000","last_clean":"2023-06-23T18:03:42.919391+0000","last_became_active":"2023-06-23T18:03:37.758859+0000","last_became_peered":"2023-06-23T18:03:37.758859+0000","last_unstale":"2023-06-23T18:03:42.919391+0000","last_undegraded":"2023-06-23T18:03:42.919391+0000","last_fullsized":"2023-06-23T18:03:42.919391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919777+0000","last_change":"2023-06-23T18:03:41.782177+0000","last_active":"2023-06-23T18:03:42.919777+0000","last_peered":"2023-06-23T18:03:42.919777+0000","last_clean":"2023-06-23T18:03:42.919777+0000","last_became_active":"2023-06-23T18:03:41.781457+0000","last_became_peered":"2023-06-23T18:03:41.781457+0000","last_unstale":"2023-06-23T18:03:42.919777+0000","last_undegraded":"2023-06-23T18:03:42.919777+0000","last_fullsized":"2023-06-23T18:03:42.919777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:38:17.449084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916697+0000","last_change":"2023-06-23T18:03:37.367776+0000","last_active":"2023-06-23T18:03:42.916697+0000","last_peered":"2023-06-23T18:03:42.916697+0000","last_clean":"2023-06-23T18:03:42.916697+0000","last_became_active":"2023-06-23T18:03:37.367288+0000","last_became_peered":"2023-06-23T18:03:37.367288+0000","last_unstale":"2023-06-23T18:03:42.916697+0000","last_undegraded":"2023-06-23T18:03:42.916697+0000","last_fullsized":"2023-06-23T18:03:42.916697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:46.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922187+0000","last_change":"2023-06-23T18:03:37.769820+0000","last_active":"2023-06-23T18:03:42.922187+0000","last_peered":"2023-06-23T18:03:42.922187+0000","last_clean":"2023-06-23T18:03:42.922187+0000","last_became_active":"2023-06-23T18:03:37.769571+0000","last_became_peered":"2023-06-23T18:03:37.769571+0000","last_unstale":"2023-06-23T18:03:42.922187+0000","last_undegraded":"2023-06-23T18:03:42.922187+0000","last_fullsized":"2023-06-23T18:03:42.922187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921931+0000","last_change":"2023-06-23T18:03:41.783182+0000","last_active":"2023-06-23T18:03:42.921931+0000","last_peered":"2023-06-23T18:03:42.921931+0000","last_clean":"2023-06-23T18:03:42.921931+0000","last_became_active":"2023-06-23T18:03:41.782753+0000","last_became_peered":"2023-06-23T18:03:41.782753+0000","last_unstale":"2023-06-23T18:03:42.921931+0000","last_undegraded":"2023-06-23T18:03:42.921931+0000","last_fullsized":"2023-06-23T18:03:42.921931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916193+0000","last_change":"2023-06-23T18:03:37.378552+0000","last_active":"2023-06-23T18:03:42.916193+0000","last_peered":"2023-06-23T18:03:42.916193+0000","last_clean":"2023-06-23T18:03:42.916193+0000","last_became_active":"2023-06-23T18:03:37.378264+0000","last_became_peered":"2023-06-23T18:03:37.378264+0000","last_unstale":"2023-06-23T18:03:42.916193+0000","last_undegraded":"2023-06-23T18:03:42.916193+0000","last_fullsized":"2023-06-23T18:03:42.916193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:04:27.977965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920059+0000","last_change":"2023-06-23T18:03:37.755621+0000","last_active":"2023-06-23T18:03:42.920059+0000","last_peered":"2023-06-23T18:03:42.920059+0000","last_clean":"2023-06-23T18:03:42.920059+0000","last_became_active":"2023-06-23T18:03:37.755341+0000","last_became_peered":"2023-06-23T18:03:37.755341+0000","last_unstale":"2023-06-23T18:03:42.920059+0000","last_undegraded":"2023-06-23T18:03:42.920059+0000","last_fullsized":"2023-06-23T18:03:42.920059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:11:22.805985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915863+0000","last_change":"2023-06-23T18:03:37.758569+0000","last_active":"2023-06-23T18:03:42.915863+0000","last_peered":"2023-06-23T18:03:42.915863+0000","last_clean":"2023-06-23T18:03:42.915863+0000","last_became_active":"2023-06-23T18:03:37.758229+0000","last_became_peered":"2023-06-23T18:03:37.758229+0000","last_unstale":"2023-06-23T18:03:42.915863+0000","last_undegraded":"2023-06-23T18:03:42.915863+0000","last_fullsized":"2023-06-23T18:03:42.915863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:56:28.950360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922336+0000","last_change":"2023-06-23T18:03:37.377384+0000","last_active":"2023-06-23T18:03:42.922336+0000","last_peered":"2023-06-23T18:03:42.922336+0000","last_clean":"2023-06-23T18:03:42.922336+0000","last_became_active":"2023-06-23T18:03:37.376940+0000","last_became_peered":"2023-06-23T18:03:37.376940+0000","last_unstale":"2023-06-23T18:03:42.922336+0000","last_undegraded":"2023-06-23T18:03:42.922336+0000","last_fullsized":"2023-06-23T18:03:42.922336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919105+0000","last_change":"2023-06-23T18:03:41.787485+0000","last_active":"2023-06-23T18:03:42.919105+0000","last_peered":"2023-06-23T18:03:42.919105+0000","last_clean":"2023-06-23T18:03:42.919105+0000","last_became_active":"2023-06-23T18:03:41.787017+0000","last_became_peered":"2023-06-23T18:03:41.787017+0000","last_unstale":"2023-06-23T18:03:42.919105+0000","last_undegraded":"2023-06-23T18:03:42.919105+0000","last_fullsized":"2023-06-23T18:03:42.919105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:48:29.879084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916767+0000","last_change":"2023-06-23T18:03:37.768665+0000","last_active":"2023-06-23T18:03:42.916767+0000","last_peered":"2023-06-23T18:03:42.916767+0000","last_clean":"2023-06-23T18:03:42.916767+0000","last_became_active":"2023-06-23T18:03:37.768373+0000","last_became_peered":"2023-06-23T18:03:37.768373+0000","last_unstale":"2023-06-23T18:03:42.916767+0000","last_undegraded":"2023-06-23T18:03:42.916767+0000","last_fullsized":"2023-06-23T18:03:42.916767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:03:33.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923756+0000","last_change":"2023-06-23T18:03:37.376353+0000","last_active":"2023-06-23T18:03:42.923756+0000","last_peered":"2023-06-23T18:03:42.923756+0000","last_clean":"2023-06-23T18:03:42.923756+0000","last_became_active":"2023-06-23T18:03:37.375931+0000","last_became_peered":"2023-06-23T18:03:37.375931+0000","last_unstale":"2023-06-23T18:03:42.923756+0000","last_undegraded":"2023-06-23T18:03:42.923756+0000","last_fullsized":"2023-06-23T18:03:42.923756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:22.517180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920338+0000","last_change":"2023-06-23T18:03:41.787494+0000","last_active":"2023-06-23T18:03:42.920338+0000","last_peered":"2023-06-23T18:03:42.920338+0000","last_clean":"2023-06-23T18:03:42.920338+0000","last_became_active":"2023-06-23T18:03:41.787099+0000","last_became_peered":"2023-06-23T18:03:41.787099+0000","last_unstale":"2023-06-23T18:03:42.920338+0000","last_undegraded":"2023-06-23T18:03:42.920338+0000","last_fullsized":"2023-06-23T18:03:42.920338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922054+0000","last_change":"2023-06-23T18:03:37.377640+0000","last_active":"2023-06-23T18:03:42.922054+0000","last_peered":"2023-06-23T18:03:42.922054+0000","last_clean":"2023-06-23T18:03:42.922054+0000","last_became_active":"2023-06-23T18:03:37.377010+0000","last_became_peered":"2023-06-23T18:03:37.377010+0000","last_unstale":"2023-06-23T18:03:42.922054+0000","last_undegraded":"2023-06-23T18:03:42.922054+0000","last_fullsized":"2023-06-23T18:03:42.922054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:48.749799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917168+0000","last_change":"2023-06-23T18:03:37.756104+0000","last_active":"2023-06-23T18:03:42.917168+0000","last_peered":"2023-06-23T18:03:42.917168+0000","last_clean":"2023-06-23T18:03:42.917168+0000","last_became_active":"2023-06-23T18:03:37.755767+0000","last_became_peered":"2023-06-23T18:03:37.755767+0000","last_unstale":"2023-06-23T18:03:42.917168+0000","last_undegraded":"2023-06-23T18:03:42.917168+0000","last_fullsized":"2023-06-23T18:03:42.917168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:22:59.783746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917308+0000","last_change":"2023-06-23T18:03:41.785136+0000","last_active":"2023-06-23T18:03:42.917308+0000","last_peered":"2023-06-23T18:03:42.917308+0000","last_clean":"2023-06-23T18:03:42.917308+0000","last_became_active":"2023-06-23T18:03:41.784711+0000","last_became_peered":"2023-06-23T18:03:41.784711+0000","last_unstale":"2023-06-23T18:03:42.917308+0000","last_undegraded":"2023-06-23T18:03:42.917308+0000","last_fullsized":"2023-06-23T18:03:42.917308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:13:19.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:47.124821+0000","last_change":"2023-06-23T18:03:37.376017+0000","last_active":"2023-06-23T18:03:47.124821+0000","last_peered":"2023-06-23T18:03:47.124821+0000","last_clean":"2023-06-23T18:03:47.124821+0000","last_became_active":"2023-06-23T18:03:37.375728+0000","last_became_peered":"2023-06-23T18:03:37.375728+0000","last_unstale":"2023-06-23T18:03:47.124821+0000","last_undegraded":"2023-06-23T18:03:47.124821+0000","last_fullsized":"2023-06-23T18:03:47.124821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:57.309841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158169+0000","last_change":"2023-06-23T18:03:37.752163+0000","last_active":"2023-06-23T18:03:42.158169+0000","last_peered":"2023-06-23T18:03:42.158169+0000","last_clean":"2023-06-23T18:03:42.158169+0000","last_became_active":"2023-06-23T18:03:37.751393+0000","last_became_peered":"2023-06-23T18:03:37.751393+0000","last_unstale":"2023-06-23T18:03:42.158169+0000","last_undegraded":"2023-06-23T18:03:42.158169+0000","last_fullsized":"2023-06-23T18:03:42.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:54:13.905688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916838+0000","last_change":"2023-06-23T18:03:41.790305+0000","last_active":"2023-06-23T18:03:42.916838+0000","last_peered":"2023-06-23T18:03:42.916838+0000","last_clean":"2023-06-23T18:03:42.916838+0000","last_became_active":"2023-06-23T18:03:41.787857+0000","last_became_peered":"2023-06-23T18:03:41.787857+0000","last_unstale":"2023-06-23T18:03:42.916838+0000","last_undegraded":"2023-06-23T18:03:42.916838+0000","last_fullsized":"2023-06-23T18:03:42.916838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:54.379340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157126+0000","last_change":"2023-06-23T18:03:37.751336+0000","last_active":"2023-06-23T18:03:42.157126+0000","last_peered":"2023-06-23T18:03:42.157126+0000","last_clean":"2023-06-23T18:03:42.157126+0000","last_became_active":"2023-06-23T18:03:37.750981+0000","last_became_peered":"2023-06-23T18:03:37.750981+0000","last_unstale":"2023-06-23T18:03:42.157126+0000","last_undegraded":"2023-06-23T18:03:42.157126+0000","last_fullsized":"2023-06-23T18:03:42.157126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:17:15.424492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920331+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:03:42.920331+0000","last_peered":"2023-06-23T18:03:42.920331+0000","last_clean":"2023-06-23T18:03:42.920331+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:03:42.920331+0000","last_undegraded":"2023-06-23T18:03:42.920331+0000","last_fullsized":"2023-06-23T18:03:42.920331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:49:09.922836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922412+0000","last_change":"2023-06-23T18:03:37.375755+0000","last_active":"2023-06-23T18:03:42.922412+0000","last_peered":"2023-06-23T18:03:42.922412+0000","last_clean":"2023-06-23T18:03:42.922412+0000","last_became_active":"2023-06-23T18:03:37.375421+0000","last_became_peered":"2023-06-23T18:03:37.375421+0000","last_unstale":"2023-06-23T18:03:42.922412+0000","last_undegraded":"2023-06-23T18:03:42.922412+0000","last_fullsized":"2023-06-23T18:03:42.922412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915864+0000","last_change":"2023-06-23T18:03:41.785345+0000","last_active":"2023-06-23T18:03:42.915864+0000","last_peered":"2023-06-23T18:03:42.915864+0000","last_clean":"2023-06-23T18:03:42.915864+0000","last_became_active":"2023-06-23T18:03:41.784105+0000","last_became_peered":"2023-06-23T18:03:41.784105+0000","last_unstale":"2023-06-23T18:03:42.915864+0000","last_undegraded":"2023-06-23T18:03:42.915864+0000","last_fullsized":"2023-06-23T18:03:42.915864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:07:51.445920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.915986+0000","last_change":"2023-06-23T18:03:37.768794+0000","last_active":"2023-06-23T18:03:42.915986+0000","last_peered":"2023-06-23T18:03:42.915986+0000","last_clean":"2023-06-23T18:03:42.915986+0000","last_became_active":"2023-06-23T18:03:37.768517+0000","last_became_peered":"2023-06-23T18:03:37.768517+0000","last_unstale":"2023-06-23T18:03:42.915986+0000","last_undegraded":"2023-06-23T18:03:42.915986+0000","last_fullsized":"2023-06-23T18:03:42.915986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:53.245906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920435+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:03:42.920435+0000","last_peered":"2023-06-23T18:03:42.920435+0000","last_clean":"2023-06-23T18:03:42.920435+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:03:42.920435+0000","last_undegraded":"2023-06-23T18:03:42.920435+0000","last_fullsized":"2023-06-23T18:03:42.920435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921748+0000","last_change":"2023-06-23T18:03:37.376156+0000","last_active":"2023-06-23T18:03:42.921748+0000","last_peered":"2023-06-23T18:03:42.921748+0000","last_clean":"2023-06-23T18:03:42.921748+0000","last_became_active":"2023-06-23T18:03:37.375870+0000","last_became_peered":"2023-06-23T18:03:37.375870+0000","last_unstale":"2023-06-23T18:03:42.921748+0000","last_undegraded":"2023-06-23T18:03:42.921748+0000","last_fullsized":"2023-06-23T18:03:42.921748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:29.342761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156557+0000","last_change":"2023-06-23T18:03:41.783782+0000","last_active":"2023-06-23T18:03:42.156557+0000","last_peered":"2023-06-23T18:03:42.156557+0000","last_clean":"2023-06-23T18:03:42.156557+0000","last_became_active":"2023-06-23T18:03:41.783336+0000","last_became_peered":"2023-06-23T18:03:41.783336+0000","last_unstale":"2023-06-23T18:03:42.156557+0000","last_undegraded":"2023-06-23T18:03:42.156557+0000","last_fullsized":"2023-06-23T18:03:42.156557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:55.511609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920320+0000","last_change":"2023-06-23T18:03:37.755707+0000","last_active":"2023-06-23T18:03:42.920320+0000","last_peered":"2023-06-23T18:03:42.920320+0000","last_clean":"2023-06-23T18:03:42.920320+0000","last_became_active":"2023-06-23T18:03:37.755413+0000","last_became_peered":"2023-06-23T18:03:37.755413+0000","last_unstale":"2023-06-23T18:03:42.920320+0000","last_undegraded":"2023-06-23T18:03:42.920320+0000","last_fullsized":"2023-06-23T18:03:42.920320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922407+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:03:42.922407+0000","last_peered":"2023-06-23T18:03:42.922407+0000","last_clean":"2023-06-23T18:03:42.922407+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:03:42.922407+0000","last_undegraded":"2023-06-23T18:03:42.922407+0000","last_fullsized":"2023-06-23T18:03:42.922407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:55.340944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918354+0000","last_change":"2023-06-23T18:03:37.370271+0000","last_active":"2023-06-23T18:03:42.918354+0000","last_peered":"2023-06-23T18:03:42.918354+0000","last_clean":"2023-06-23T18:03:42.918354+0000","last_became_active":"2023-06-23T18:03:37.369994+0000","last_became_peered":"2023-06-23T18:03:37.369994+0000","last_unstale":"2023-06-23T18:03:42.918354+0000","last_undegraded":"2023-06-23T18:03:42.918354+0000","last_fullsized":"2023-06-23T18:03:42.918354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:36.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920193+0000","last_change":"2023-06-23T18:03:41.793568+0000","last_active":"2023-06-23T18:03:42.920193+0000","last_peered":"2023-06-23T18:03:42.920193+0000","last_clean":"2023-06-23T18:03:42.920193+0000","last_became_active":"2023-06-23T18:03:41.793190+0000","last_became_peered":"2023-06-23T18:03:41.793190+0000","last_unstale":"2023-06-23T18:03:42.920193+0000","last_undegraded":"2023-06-23T18:03:42.920193+0000","last_fullsized":"2023-06-23T18:03:42.920193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:35.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917232+0000","last_change":"2023-06-23T18:03:37.764627+0000","last_active":"2023-06-23T18:03:42.917232+0000","last_peered":"2023-06-23T18:03:42.917232+0000","last_clean":"2023-06-23T18:03:42.917232+0000","last_became_active":"2023-06-23T18:03:37.764255+0000","last_became_peered":"2023-06-23T18:03:37.764255+0000","last_unstale":"2023-06-23T18:03:42.917232+0000","last_undegraded":"2023-06-23T18:03:42.917232+0000","last_fullsized":"2023-06-23T18:03:42.917232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:20.856505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918569+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:03:42.918569+0000","last_peered":"2023-06-23T18:03:42.918569+0000","last_clean":"2023-06-23T18:03:42.918569+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:03:42.918569+0000","last_undegraded":"2023-06-23T18:03:42.918569+0000","last_fullsized":"2023-06-23T18:03:42.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:25.233307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924293+0000","last_change":"2023-06-23T18:03:37.371127+0000","last_active":"2023-06-23T18:03:42.924293+0000","last_peered":"2023-06-23T18:03:42.924293+0000","last_clean":"2023-06-23T18:03:42.924293+0000","last_became_active":"2023-06-23T18:03:37.370810+0000","last_became_peered":"2023-06-23T18:03:37.370810+0000","last_unstale":"2023-06-23T18:03:42.924293+0000","last_undegraded":"2023-06-23T18:03:42.924293+0000","last_fullsized":"2023-06-23T18:03:42.924293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:15:42.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919827+0000","last_change":"2023-06-23T18:03:41.791645+0000","last_active":"2023-06-23T18:03:42.919827+0000","last_peered":"2023-06-23T18:03:42.919827+0000","last_clean":"2023-06-23T18:03:42.919827+0000","last_became_active":"2023-06-23T18:03:41.791276+0000","last_became_peered":"2023-06-23T18:03:41.791276+0000","last_unstale":"2023-06-23T18:03:42.919827+0000","last_undegraded":"2023-06-23T18:03:42.919827+0000","last_fullsized":"2023-06-23T18:03:42.919827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:24.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918088+0000","last_change":"2023-06-23T18:03:37.754565+0000","last_active":"2023-06-23T18:03:42.918088+0000","last_peered":"2023-06-23T18:03:42.918088+0000","last_clean":"2023-06-23T18:03:42.918088+0000","last_became_active":"2023-06-23T18:03:37.754281+0000","last_became_peered":"2023-06-23T18:03:37.754281+0000","last_unstale":"2023-06-23T18:03:42.918088+0000","last_undegraded":"2023-06-23T18:03:42.918088+0000","last_fullsized":"2023-06-23T18:03:42.918088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158267+0000","last_change":"2023-06-23T18:03:37.381149+0000","last_active":"2023-06-23T18:03:42.158267+0000","last_peered":"2023-06-23T18:03:42.158267+0000","last_clean":"2023-06-23T18:03:42.158267+0000","last_became_active":"2023-06-23T18:03:37.380831+0000","last_became_peered":"2023-06-23T18:03:37.380831+0000","last_unstale":"2023-06-23T18:03:42.158267+0000","last_undegraded":"2023-06-23T18:03:42.158267+0000","last_fullsized":"2023-06-23T18:03:42.158267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:40:27.306991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923558+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:03:42.923558+0000","last_peered":"2023-06-23T18:03:42.923558+0000","last_clean":"2023-06-23T18:03:42.923558+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:03:42.923558+0000","last_undegraded":"2023-06-23T18:03:42.923558+0000","last_fullsized":"2023-06-23T18:03:42.923558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921567+0000","last_change":"2023-06-23T18:03:41.787633+0000","last_active":"2023-06-23T18:03:42.921567+0000","last_peered":"2023-06-23T18:03:42.921567+0000","last_clean":"2023-06-23T18:03:42.921567+0000","last_became_active":"2023-06-23T18:03:41.786906+0000","last_became_peered":"2023-06-23T18:03:41.786906+0000","last_unstale":"2023-06-23T18:03:42.921567+0000","last_undegraded":"2023-06-23T18:03:42.921567+0000","last_fullsized":"2023-06-23T18:03:42.921567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:11.457412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919645+0000","last_change":"2023-06-23T18:03:37.764949+0000","last_active":"2023-06-23T18:03:42.919645+0000","last_peered":"2023-06-23T18:03:42.919645+0000","last_clean":"2023-06-23T18:03:42.919645+0000","last_became_active":"2023-06-23T18:03:37.764358+0000","last_became_peered":"2023-06-23T18:03:37.764358+0000","last_unstale":"2023-06-23T18:03:42.919645+0000","last_undegraded":"2023-06-23T18:03:42.919645+0000","last_fullsized":"2023-06-23T18:03:42.919645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916086+0000","last_change":"2023-06-23T18:03:37.371283+0000","last_active":"2023-06-23T18:03:42.916086+0000","last_peered":"2023-06-23T18:03:42.916086+0000","last_clean":"2023-06-23T18:03:42.916086+0000","last_became_active":"2023-06-23T18:03:37.370358+0000","last_became_peered":"2023-06-23T18:03:37.370358+0000","last_unstale":"2023-06-23T18:03:42.916086+0000","last_undegraded":"2023-06-23T18:03:42.916086+0000","last_fullsized":"2023-06-23T18:03:42.916086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152204+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:03:42.152204+0000","last_peered":"2023-06-23T18:03:42.152204+0000","last_clean":"2023-06-23T18:03:42.152204+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:03:42.152204+0000","last_undegraded":"2023-06-23T18:03:42.152204+0000","last_fullsized":"2023-06-23T18:03:42.152204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:44.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152495+0000","last_change":"2023-06-23T18:03:41.787323+0000","last_active":"2023-06-23T18:03:42.152495+0000","last_peered":"2023-06-23T18:03:42.152495+0000","last_clean":"2023-06-23T18:03:42.152495+0000","last_became_active":"2023-06-23T18:03:41.786742+0000","last_became_peered":"2023-06-23T18:03:41.786742+0000","last_unstale":"2023-06-23T18:03:42.152495+0000","last_undegraded":"2023-06-23T18:03:42.152495+0000","last_fullsized":"2023-06-23T18:03:42.152495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:39:47.931343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922227+0000","last_change":"2023-06-23T18:03:37.743506+0000","last_active":"2023-06-23T18:03:42.922227+0000","last_peered":"2023-06-23T18:03:42.922227+0000","last_clean":"2023-06-23T18:03:42.922227+0000","last_became_active":"2023-06-23T18:03:37.743222+0000","last_became_peered":"2023-06-23T18:03:37.743222+0000","last_unstale":"2023-06-23T18:03:42.922227+0000","last_undegraded":"2023-06-23T18:03:42.922227+0000","last_fullsized":"2023-06-23T18:03:42.922227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:43.133522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920271+0000","last_change":"2023-06-23T18:03:37.375782+0000","last_active":"2023-06-23T18:03:42.920271+0000","last_peered":"2023-06-23T18:03:42.920271+0000","last_clean":"2023-06-23T18:03:42.920271+0000","last_became_active":"2023-06-23T18:03:37.375451+0000","last_became_peered":"2023-06-23T18:03:37.375451+0000","last_unstale":"2023-06-23T18:03:42.920271+0000","last_undegraded":"2023-06-23T18:03:42.920271+0000","last_fullsized":"2023-06-23T18:03:42.920271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:35.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921282+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:03:42.921282+0000","last_peered":"2023-06-23T18:03:42.921282+0000","last_clean":"2023-06-23T18:03:42.921282+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:03:42.921282+0000","last_undegraded":"2023-06-23T18:03:42.921282+0000","last_fullsized":"2023-06-23T18:03:42.921282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:52.910752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922998+0000","last_change":"2023-06-23T18:03:41.788944+0000","last_active":"2023-06-23T18:03:42.922998+0000","last_peered":"2023-06-23T18:03:42.922998+0000","last_clean":"2023-06-23T18:03:42.922998+0000","last_became_active":"2023-06-23T18:03:41.785916+0000","last_became_peered":"2023-06-23T18:03:41.785916+0000","last_unstale":"2023-06-23T18:03:42.922998+0000","last_undegraded":"2023-06-23T18:03:42.922998+0000","last_fullsized":"2023-06-23T18:03:42.922998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919919+0000","last_change":"2023-06-23T18:03:37.750749+0000","last_active":"2023-06-23T18:03:42.919919+0000","last_peered":"2023-06-23T18:03:42.919919+0000","last_clean":"2023-06-23T18:03:42.919919+0000","last_became_active":"2023-06-23T18:03:37.750424+0000","last_became_peered":"2023-06-23T18:03:37.750424+0000","last_unstale":"2023-06-23T18:03:42.919919+0000","last_undegraded":"2023-06-23T18:03:42.919919+0000","last_fullsized":"2023-06-23T18:03:42.919919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:10.920865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917678+0000","last_change":"2023-06-23T18:03:37.377053+0000","last_active":"2023-06-23T18:03:42.917678+0000","last_peered":"2023-06-23T18:03:42.917678+0000","last_clean":"2023-06-23T18:03:42.917678+0000","last_became_active":"2023-06-23T18:03:37.376770+0000","last_became_peered":"2023-06-23T18:03:37.376770+0000","last_unstale":"2023-06-23T18:03:42.917678+0000","last_undegraded":"2023-06-23T18:03:42.917678+0000","last_fullsized":"2023-06-23T18:03:42.917678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:48:22.071155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923092+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:03:42.923092+0000","last_peered":"2023-06-23T18:03:42.923092+0000","last_clean":"2023-06-23T18:03:42.923092+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:03:42.923092+0000","last_undegraded":"2023-06-23T18:03:42.923092+0000","last_fullsized":"2023-06-23T18:03:42.923092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:00.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921588+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:03:42.921588+0000","last_peered":"2023-06-23T18:03:42.921588+0000","last_clean":"2023-06-23T18:03:42.921588+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:03:42.921588+0000","last_undegraded":"2023-06-23T18:03:42.921588+0000","last_fullsized":"2023-06-23T18:03:42.921588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:15:38.015240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921628+0000","last_change":"2023-06-23T18:03:41.792372+0000","last_active":"2023-06-23T18:03:42.921628+0000","last_peered":"2023-06-23T18:03:42.921628+0000","last_clean":"2023-06-23T18:03:42.921628+0000","last_became_active":"2023-06-23T18:03:41.791937+0000","last_became_peered":"2023-06-23T18:03:41.791937+0000","last_unstale":"2023-06-23T18:03:42.921628+0000","last_undegraded":"2023-06-23T18:03:42.921628+0000","last_fullsized":"2023-06-23T18:03:42.921628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:26.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919683+0000","last_change":"2023-06-23T18:03:37.764721+0000","last_active":"2023-06-23T18:03:42.919683+0000","last_peered":"2023-06-23T18:03:42.919683+0000","last_clean":"2023-06-23T18:03:42.919683+0000","last_became_active":"2023-06-23T18:03:37.755578+0000","last_became_peered":"2023-06-23T18:03:37.755578+0000","last_unstale":"2023-06-23T18:03:42.919683+0000","last_undegraded":"2023-06-23T18:03:42.919683+0000","last_fullsized":"2023-06-23T18:03:42.919683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916131+0000","last_change":"2023-06-23T18:03:37.369824+0000","last_active":"2023-06-23T18:03:42.916131+0000","last_peered":"2023-06-23T18:03:42.916131+0000","last_clean":"2023-06-23T18:03:42.916131+0000","last_became_active":"2023-06-23T18:03:37.369094+0000","last_became_peered":"2023-06-23T18:03:37.369094+0000","last_unstale":"2023-06-23T18:03:42.916131+0000","last_undegraded":"2023-06-23T18:03:42.916131+0000","last_fullsized":"2023-06-23T18:03:42.916131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:53.895398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159093+0000","last_change":"2023-06-23T18:03:41.789799+0000","last_active":"2023-06-23T18:03:42.159093+0000","last_peered":"2023-06-23T18:03:42.159093+0000","last_clean":"2023-06-23T18:03:42.159093+0000","last_became_active":"2023-06-23T18:03:41.789426+0000","last_became_peered":"2023-06-23T18:03:41.789426+0000","last_unstale":"2023-06-23T18:03:42.159093+0000","last_undegraded":"2023-06-23T18:03:42.159093+0000","last_fullsized":"2023-06-23T18:03:42.159093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:36:43.668342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.159103+0000","last_change":"2023-06-23T18:03:37.751763+0000","last_active":"2023-06-23T18:03:42.159103+0000","last_peered":"2023-06-23T18:03:42.159103+0000","last_clean":"2023-06-23T18:03:42.159103+0000","last_became_active":"2023-06-23T18:03:37.751310+0000","last_became_peered":"2023-06-23T18:03:37.751310+0000","last_unstale":"2023-06-23T18:03:42.159103+0000","last_undegraded":"2023-06-23T18:03:42.159103+0000","last_fullsized":"2023-06-23T18:03:42.159103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:00.581207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157189+0000","last_change":"2023-06-23T18:03:37.373425+0000","last_active":"2023-06-23T18:03:42.157189+0000","last_peered":"2023-06-23T18:03:42.157189+0000","last_clean":"2023-06-23T18:03:42.157189+0000","last_became_active":"2023-06-23T18:03:37.373154+0000","last_became_peered":"2023-06-23T18:03:37.373154+0000","last_unstale":"2023-06-23T18:03:42.157189+0000","last_undegraded":"2023-06-23T18:03:42.157189+0000","last_fullsized":"2023-06-23T18:03:42.157189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:33.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922639+0000","last_change":"2023-06-23T18:03:41.792127+0000","last_active":"2023-06-23T18:03:42.922639+0000","last_peered":"2023-06-23T18:03:42.922639+0000","last_clean":"2023-06-23T18:03:42.922639+0000","last_became_active":"2023-06-23T18:03:41.789481+0000","last_became_peered":"2023-06-23T18:03:41.789481+0000","last_unstale":"2023-06-23T18:03:42.922639+0000","last_undegraded":"2023-06-23T18:03:42.922639+0000","last_fullsized":"2023-06-23T18:03:42.922639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158754+0000","last_change":"2023-06-23T18:03:37.754662+0000","last_active":"2023-06-23T18:03:42.158754+0000","last_peered":"2023-06-23T18:03:42.158754+0000","last_clean":"2023-06-23T18:03:42.158754+0000","last_became_active":"2023-06-23T18:03:37.753982+0000","last_became_peered":"2023-06-23T18:03:37.753982+0000","last_unstale":"2023-06-23T18:03:42.158754+0000","last_undegraded":"2023-06-23T18:03:42.158754+0000","last_fullsized":"2023-06-23T18:03:42.158754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:36:40.534179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921128+0000","last_change":"2023-06-23T18:03:37.380222+0000","last_active":"2023-06-23T18:03:42.921128+0000","last_peered":"2023-06-23T18:03:42.921128+0000","last_clean":"2023-06-23T18:03:42.921128+0000","last_became_active":"2023-06-23T18:03:37.379866+0000","last_became_peered":"2023-06-23T18:03:37.379866+0000","last_unstale":"2023-06-23T18:03:42.921128+0000","last_undegraded":"2023-06-23T18:03:42.921128+0000","last_fullsized":"2023-06-23T18:03:42.921128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:12.333579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917855+0000","last_change":"2023-06-23T18:03:41.782310+0000","last_active":"2023-06-23T18:03:42.917855+0000","last_peered":"2023-06-23T18:03:42.917855+0000","last_clean":"2023-06-23T18:03:42.917855+0000","last_became_active":"2023-06-23T18:03:41.781820+0000","last_became_peered":"2023-06-23T18:03:41.781820+0000","last_unstale":"2023-06-23T18:03:42.917855+0000","last_undegraded":"2023-06-23T18:03:42.917855+0000","last_fullsized":"2023-06-23T18:03:42.917855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:39.845448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158787+0000","last_change":"2023-06-23T18:03:37.753742+0000","last_active":"2023-06-23T18:03:42.158787+0000","last_peered":"2023-06-23T18:03:42.158787+0000","last_clean":"2023-06-23T18:03:42.158787+0000","last_became_active":"2023-06-23T18:03:37.753391+0000","last_became_peered":"2023-06-23T18:03:37.753391+0000","last_unstale":"2023-06-23T18:03:42.158787+0000","last_undegraded":"2023-06-23T18:03:42.158787+0000","last_fullsized":"2023-06-23T18:03:42.158787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:54.787520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918436+0000","last_change":"2023-06-23T18:03:37.378501+0000","last_active":"2023-06-23T18:03:42.918436+0000","last_peered":"2023-06-23T18:03:42.918436+0000","last_clean":"2023-06-23T18:03:42.918436+0000","last_became_active":"2023-06-23T18:03:37.378222+0000","last_became_peered":"2023-06-23T18:03:37.378222+0000","last_unstale":"2023-06-23T18:03:42.918436+0000","last_undegraded":"2023-06-23T18:03:42.918436+0000","last_fullsized":"2023-06-23T18:03:42.918436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:49.965045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152626+0000","last_change":"2023-06-23T18:03:41.788999+0000","last_active":"2023-06-23T18:03:42.152626+0000","last_peered":"2023-06-23T18:03:42.152626+0000","last_clean":"2023-06-23T18:03:42.152626+0000","last_became_active":"2023-06-23T18:03:41.788606+0000","last_became_peered":"2023-06-23T18:03:41.788606+0000","last_unstale":"2023-06-23T18:03:42.152626+0000","last_undegraded":"2023-06-23T18:03:42.152626+0000","last_fullsized":"2023-06-23T18:03:42.152626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:40:16.265229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920805+0000","last_change":"2023-06-23T18:03:37.751267+0000","last_active":"2023-06-23T18:03:42.920805+0000","last_peered":"2023-06-23T18:03:42.920805+0000","last_clean":"2023-06-23T18:03:42.920805+0000","last_became_active":"2023-06-23T18:03:37.746037+0000","last_became_peered":"2023-06-23T18:03:37.746037+0000","last_unstale":"2023-06-23T18:03:42.920805+0000","last_undegraded":"2023-06-23T18:03:42.920805+0000","last_fullsized":"2023-06-23T18:03:42.920805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922834+0000","last_change":"2023-06-23T18:03:37.369505+0000","last_active":"2023-06-23T18:03:42.922834+0000","last_peered":"2023-06-23T18:03:42.922834+0000","last_clean":"2023-06-23T18:03:42.922834+0000","last_became_active":"2023-06-23T18:03:37.369174+0000","last_became_peered":"2023-06-23T18:03:37.369174+0000","last_unstale":"2023-06-23T18:03:42.922834+0000","last_undegraded":"2023-06-23T18:03:42.922834+0000","last_fullsized":"2023-06-23T18:03:42.922834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151984+0000","last_change":"2023-06-23T18:03:41.781321+0000","last_active":"2023-06-23T18:03:42.151984+0000","last_peered":"2023-06-23T18:03:42.151984+0000","last_clean":"2023-06-23T18:03:42.151984+0000","last_became_active":"2023-06-23T18:03:41.780831+0000","last_became_peered":"2023-06-23T18:03:41.780831+0000","last_unstale":"2023-06-23T18:03:42.151984+0000","last_undegraded":"2023-06-23T18:03:42.151984+0000","last_fullsized":"2023-06-23T18:03:42.151984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:53.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920510+0000","last_change":"2023-06-23T18:03:37.761108+0000","last_active":"2023-06-23T18:03:42.920510+0000","last_peered":"2023-06-23T18:03:42.920510+0000","last_clean":"2023-06-23T18:03:42.920510+0000","last_became_active":"2023-06-23T18:03:37.760041+0000","last_became_peered":"2023-06-23T18:03:37.760041+0000","last_unstale":"2023-06-23T18:03:42.920510+0000","last_undegraded":"2023-06-23T18:03:42.920510+0000","last_fullsized":"2023-06-23T18:03:42.920510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:46:38.396933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922050+0000","last_change":"2023-06-23T18:03:37.377838+0000","last_active":"2023-06-23T18:03:42.922050+0000","last_peered":"2023-06-23T18:03:42.922050+0000","last_clean":"2023-06-23T18:03:42.922050+0000","last_became_active":"2023-06-23T18:03:37.377381+0000","last_became_peered":"2023-06-23T18:03:37.377381+0000","last_unstale":"2023-06-23T18:03:42.922050+0000","last_undegraded":"2023-06-23T18:03:42.922050+0000","last_fullsized":"2023-06-23T18:03:42.922050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:26.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152297+0000","last_change":"2023-06-23T18:03:41.787926+0000","last_active":"2023-06-23T18:03:42.152297+0000","last_peered":"2023-06-23T18:03:42.152297+0000","last_clean":"2023-06-23T18:03:42.152297+0000","last_became_active":"2023-06-23T18:03:41.787539+0000","last_became_peered":"2023-06-23T18:03:41.787539+0000","last_unstale":"2023-06-23T18:03:42.152297+0000","last_undegraded":"2023-06-23T18:03:42.152297+0000","last_fullsized":"2023-06-23T18:03:42.152297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:58:49.966258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156824+0000","last_change":"2023-06-23T18:03:37.745296+0000","last_active":"2023-06-23T18:03:42.156824+0000","last_peered":"2023-06-23T18:03:42.156824+0000","last_clean":"2023-06-23T18:03:42.156824+0000","last_became_active":"2023-06-23T18:03:37.744996+0000","last_became_peered":"2023-06-23T18:03:37.744996+0000","last_unstale":"2023-06-23T18:03:42.156824+0000","last_undegraded":"2023-06-23T18:03:42.156824+0000","last_fullsized":"2023-06-23T18:03:42.156824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:07:37.054289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922271+0000","last_change":"2023-06-23T18:03:37.381405+0000","last_active":"2023-06-23T18:03:42.922271+0000","last_peered":"2023-06-23T18:03:42.922271+0000","last_clean":"2023-06-23T18:03:42.922271+0000","last_became_active":"2023-06-23T18:03:37.381038+0000","last_became_peered":"2023-06-23T18:03:37.381038+0000","last_unstale":"2023-06-23T18:03:42.922271+0000","last_undegraded":"2023-06-23T18:03:42.922271+0000","last_fullsized":"2023-06-23T18:03:42.922271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:48:14.127704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.151761+0000","last_change":"2023-06-23T18:03:41.787211+0000","last_active":"2023-06-23T18:03:42.151761+0000","last_peered":"2023-06-23T18:03:42.151761+0000","last_clean":"2023-06-23T18:03:42.151761+0000","last_became_active":"2023-06-23T18:03:41.786809+0000","last_became_peered":"2023-06-23T18:03:41.786809+0000","last_unstale":"2023-06-23T18:03:42.151761+0000","last_undegraded":"2023-06-23T18:03:42.151761+0000","last_fullsized":"2023-06-23T18:03:42.151761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:01:42.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919723+0000","last_change":"2023-06-23T18:03:37.770649+0000","last_active":"2023-06-23T18:03:42.919723+0000","last_peered":"2023-06-23T18:03:42.919723+0000","last_clean":"2023-06-23T18:03:42.919723+0000","last_became_active":"2023-06-23T18:03:37.770326+0000","last_became_peered":"2023-06-23T18:03:37.770326+0000","last_unstale":"2023-06-23T18:03:42.919723+0000","last_undegraded":"2023-06-23T18:03:42.919723+0000","last_fullsized":"2023-06-23T18:03:42.919723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923872+0000","last_change":"2023-06-23T18:03:37.381617+0000","last_active":"2023-06-23T18:03:42.923872+0000","last_peered":"2023-06-23T18:03:42.923872+0000","last_clean":"2023-06-23T18:03:42.923872+0000","last_became_active":"2023-06-23T18:03:37.381342+0000","last_became_peered":"2023-06-23T18:03:37.381342+0000","last_unstale":"2023-06-23T18:03:42.923872+0000","last_undegraded":"2023-06-23T18:03:42.923872+0000","last_fullsized":"2023-06-23T18:03:42.923872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:31.692075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920724+0000","last_change":"2023-06-23T18:03:41.791679+0000","last_active":"2023-06-23T18:03:42.920724+0000","last_peered":"2023-06-23T18:03:42.920724+0000","last_clean":"2023-06-23T18:03:42.920724+0000","last_became_active":"2023-06-23T18:03:41.788101+0000","last_became_peered":"2023-06-23T18:03:41.788101+0000","last_unstale":"2023-06-23T18:03:42.920724+0000","last_undegraded":"2023-06-23T18:03:42.920724+0000","last_fullsized":"2023-06-23T18:03:42.920724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:40.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924128+0000","last_change":"2023-06-23T18:03:41.789672+0000","last_active":"2023-06-23T18:03:42.924128+0000","last_peered":"2023-06-23T18:03:42.924128+0000","last_clean":"2023-06-23T18:03:42.924128+0000","last_became_active":"2023-06-23T18:03:41.789258+0000","last_became_peered":"2023-06-23T18:03:41.789258+0000","last_unstale":"2023-06-23T18:03:42.924128+0000","last_undegraded":"2023-06-23T18:03:42.924128+0000","last_fullsized":"2023-06-23T18:03:42.924128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:35:03.864692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919206+0000","last_change":"2023-06-23T18:03:37.380523+0000","last_active":"2023-06-23T18:03:42.919206+0000","last_peered":"2023-06-23T18:03:42.919206+0000","last_clean":"2023-06-23T18:03:42.919206+0000","last_became_active":"2023-06-23T18:03:37.380210+0000","last_became_peered":"2023-06-23T18:03:37.380210+0000","last_unstale":"2023-06-23T18:03:42.919206+0000","last_undegraded":"2023-06-23T18:03:42.919206+0000","last_fullsized":"2023-06-23T18:03:42.919206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:27:57.050399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921568+0000","last_change":"2023-06-23T18:03:37.769174+0000","last_active":"2023-06-23T18:03:42.921568+0000","last_peered":"2023-06-23T18:03:42.921568+0000","last_clean":"2023-06-23T18:03:42.921568+0000","last_became_active":"2023-06-23T18:03:37.768922+0000","last_became_peered":"2023-06-23T18:03:37.768922+0000","last_unstale":"2023-06-23T18:03:42.921568+0000","last_undegraded":"2023-06-23T18:03:42.921568+0000","last_fullsized":"2023-06-23T18:03:42.921568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:42:37.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921045+0000","last_change":"2023-06-23T18:03:41.785038+0000","last_active":"2023-06-23T18:03:42.921045+0000","last_peered":"2023-06-23T18:03:42.921045+0000","last_clean":"2023-06-23T18:03:42.921045+0000","last_became_active":"2023-06-23T18:03:41.784124+0000","last_became_peered":"2023-06-23T18:03:41.784124+0000","last_unstale":"2023-06-23T18:03:42.921045+0000","last_undegraded":"2023-06-23T18:03:42.921045+0000","last_fullsized":"2023-06-23T18:03:42.921045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:04:01.963233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921205+0000","last_change":"2023-06-23T18:03:37.381450+0000","last_active":"2023-06-23T18:03:42.921205+0000","last_peered":"2023-06-23T18:03:42.921205+0000","last_clean":"2023-06-23T18:03:42.921205+0000","last_became_active":"2023-06-23T18:03:37.381137+0000","last_became_peered":"2023-06-23T18:03:37.381137+0000","last_unstale":"2023-06-23T18:03:42.921205+0000","last_undegraded":"2023-06-23T18:03:42.921205+0000","last_fullsized":"2023-06-23T18:03:42.921205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:04.690816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922338+0000","last_change":"2023-06-23T18:03:37.752538+0000","last_active":"2023-06-23T18:03:42.922338+0000","last_peered":"2023-06-23T18:03:42.922338+0000","last_clean":"2023-06-23T18:03:42.922338+0000","last_became_active":"2023-06-23T18:03:37.752276+0000","last_became_peered":"2023-06-23T18:03:37.752276+0000","last_unstale":"2023-06-23T18:03:42.922338+0000","last_undegraded":"2023-06-23T18:03:42.922338+0000","last_fullsized":"2023-06-23T18:03:42.922338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:55:23.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921015+0000","last_change":"2023-06-23T18:03:37.376067+0000","last_active":"2023-06-23T18:03:42.921015+0000","last_peered":"2023-06-23T18:03:42.921015+0000","last_clean":"2023-06-23T18:03:42.921015+0000","last_became_active":"2023-06-23T18:03:37.375582+0000","last_became_peered":"2023-06-23T18:03:37.375582+0000","last_unstale":"2023-06-23T18:03:42.921015+0000","last_undegraded":"2023-06-23T18:03:42.921015+0000","last_fullsized":"2023-06-23T18:03:42.921015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:34.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924004+0000","last_change":"2023-06-23T18:03:37.770794+0000","last_active":"2023-06-23T18:03:42.924004+0000","last_peered":"2023-06-23T18:03:42.924004+0000","last_clean":"2023-06-23T18:03:42.924004+0000","last_became_active":"2023-06-23T18:03:37.770508+0000","last_became_peered":"2023-06-23T18:03:37.770508+0000","last_unstale":"2023-06-23T18:03:42.924004+0000","last_undegraded":"2023-06-23T18:03:42.924004+0000","last_fullsized":"2023-06-23T18:03:42.924004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:53:27.966872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919085+0000","last_change":"2023-06-23T18:03:41.784964+0000","last_active":"2023-06-23T18:03:42.919085+0000","last_peered":"2023-06-23T18:03:42.919085+0000","last_clean":"2023-06-23T18:03:42.919085+0000","last_became_active":"2023-06-23T18:03:41.783875+0000","last_became_peered":"2023-06-23T18:03:41.783875+0000","last_unstale":"2023-06-23T18:03:42.919085+0000","last_undegraded":"2023-06-23T18:03:42.919085+0000","last_fullsized":"2023-06-23T18:03:42.919085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:03.189212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921280+0000","last_change":"2023-06-23T18:03:37.381471+0000","last_active":"2023-06-23T18:03:42.921280+0000","last_peered":"2023-06-23T18:03:42.921280+0000","last_clean":"2023-06-23T18:03:42.921280+0000","last_became_active":"2023-06-23T18:03:37.381080+0000","last_became_peered":"2023-06-23T18:03:37.381080+0000","last_unstale":"2023-06-23T18:03:42.921280+0000","last_undegraded":"2023-06-23T18:03:42.921280+0000","last_fullsized":"2023-06-23T18:03:42.921280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:36.331028+0000","last_clean_scrub_stamp":"2023-06-23T18:03:36.331028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:36.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919682+0000","last_change":"2023-06-23T18:03:37.770191+0000","last_active":"2023-06-23T18:03:42.919682+0000","last_peered":"2023-06-23T18:03:42.919682+0000","last_clean":"2023-06-23T18:03:42.919682+0000","last_became_active":"2023-06-23T18:03:37.769918+0000","last_became_peered":"2023-06-23T18:03:37.769918+0000","last_unstale":"2023-06-23T18:03:42.919682+0000","last_undegraded":"2023-06-23T18:03:42.919682+0000","last_fullsized":"2023-06-23T18:03:42.919682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:59:36.439982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.924483+0000","last_change":"2023-06-23T18:03:41.789295+0000","last_active":"2023-06-23T18:03:42.924483+0000","last_peered":"2023-06-23T18:03:42.924483+0000","last_clean":"2023-06-23T18:03:42.924483+0000","last_became_active":"2023-06-23T18:03:41.788890+0000","last_became_peered":"2023-06-23T18:03:41.788890+0000","last_unstale":"2023-06-23T18:03:42.924483+0000","last_undegraded":"2023-06-23T18:03:42.924483+0000","last_fullsized":"2023-06-23T18:03:42.924483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:40.745322+0000","last_clean_scrub_stamp":"2023-06-23T18:03:40.745322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:55:36.106369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918865+0000","last_change":"2023-06-23T18:03:37.759952+0000","last_active":"2023-06-23T18:03:42.918865+0000","last_peered":"2023-06-23T18:03:42.918865+0000","last_clean":"2023-06-23T18:03:42.918865+0000","last_became_active":"2023-06-23T18:03:37.759685+0000","last_became_peered":"2023-06-23T18:03:37.759685+0000","last_unstale":"2023-06-23T18:03:42.918865+0000","last_undegraded":"2023-06-23T18:03:42.918865+0000","last_fullsized":"2023-06-23T18:03:42.918865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:01:11.356694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916611+0000","last_change":"2023-06-23T18:03:37.761169+0000","last_active":"2023-06-23T18:03:42.916611+0000","last_peered":"2023-06-23T18:03:42.916611+0000","last_clean":"2023-06-23T18:03:42.916611+0000","last_became_active":"2023-06-23T18:03:37.760915+0000","last_became_peered":"2023-06-23T18:03:37.760915+0000","last_unstale":"2023-06-23T18:03:42.916611+0000","last_undegraded":"2023-06-23T18:03:42.916611+0000","last_fullsized":"2023-06-23T18:03:42.916611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:15:41.693241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157660+0000","last_change":"2023-06-23T18:03:37.756608+0000","last_active":"2023-06-23T18:03:42.157660+0000","last_peered":"2023-06-23T18:03:42.157660+0000","last_clean":"2023-06-23T18:03:42.157660+0000","last_became_active":"2023-06-23T18:03:37.756267+0000","last_became_peered":"2023-06-23T18:03:37.756267+0000","last_unstale":"2023-06-23T18:03:42.157660+0000","last_undegraded":"2023-06-23T18:03:42.157660+0000","last_fullsized":"2023-06-23T18:03:42.157660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:39:56.896505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.917984+0000","last_change":"2023-06-23T18:03:37.768762+0000","last_active":"2023-06-23T18:03:42.917984+0000","last_peered":"2023-06-23T18:03:42.917984+0000","last_clean":"2023-06-23T18:03:42.917984+0000","last_became_active":"2023-06-23T18:03:37.768493+0000","last_became_peered":"2023-06-23T18:03:37.768493+0000","last_unstale":"2023-06-23T18:03:42.917984+0000","last_undegraded":"2023-06-23T18:03:42.917984+0000","last_fullsized":"2023-06-23T18:03:42.917984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:38:47.275516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.918340+0000","last_change":"2023-06-23T18:03:37.770023+0000","last_active":"2023-06-23T18:03:42.918340+0000","last_peered":"2023-06-23T18:03:42.918340+0000","last_clean":"2023-06-23T18:03:42.918340+0000","last_became_active":"2023-06-23T18:03:37.769776+0000","last_became_peered":"2023-06-23T18:03:37.769776+0000","last_unstale":"2023-06-23T18:03:42.918340+0000","last_undegraded":"2023-06-23T18:03:42.918340+0000","last_fullsized":"2023-06-23T18:03:42.918340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923845+0000","last_change":"2023-06-23T18:03:37.772405+0000","last_active":"2023-06-23T18:03:42.923845+0000","last_peered":"2023-06-23T18:03:42.923845+0000","last_clean":"2023-06-23T18:03:42.923845+0000","last_became_active":"2023-06-23T18:03:37.772086+0000","last_became_peered":"2023-06-23T18:03:37.772086+0000","last_unstale":"2023-06-23T18:03:42.923845+0000","last_undegraded":"2023-06-23T18:03:42.923845+0000","last_fullsized":"2023-06-23T18:03:42.923845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:30:49.758628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919917+0000","last_change":"2023-06-23T18:03:37.768948+0000","last_active":"2023-06-23T18:03:42.919917+0000","last_peered":"2023-06-23T18:03:42.919917+0000","last_clean":"2023-06-23T18:03:42.919917+0000","last_became_active":"2023-06-23T18:03:37.768355+0000","last_became_peered":"2023-06-23T18:03:37.768355+0000","last_unstale":"2023-06-23T18:03:42.919917+0000","last_undegraded":"2023-06-23T18:03:42.919917+0000","last_fullsized":"2023-06-23T18:03:42.919917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919527+0000","last_change":"2023-06-23T18:03:37.764486+0000","last_active":"2023-06-23T18:03:42.919527+0000","last_peered":"2023-06-23T18:03:42.919527+0000","last_clean":"2023-06-23T18:03:42.919527+0000","last_became_active":"2023-06-23T18:03:37.755419+0000","last_became_peered":"2023-06-23T18:03:37.755419+0000","last_unstale":"2023-06-23T18:03:42.919527+0000","last_undegraded":"2023-06-23T18:03:42.919527+0000","last_fullsized":"2023-06-23T18:03:42.919527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921001+0000","last_change":"2023-06-23T18:03:37.770648+0000","last_active":"2023-06-23T18:03:42.921001+0000","last_peered":"2023-06-23T18:03:42.921001+0000","last_clean":"2023-06-23T18:03:42.921001+0000","last_became_active":"2023-06-23T18:03:37.770271+0000","last_became_peered":"2023-06-23T18:03:37.770271+0000","last_unstale":"2023-06-23T18:03:42.921001+0000","last_undegraded":"2023-06-23T18:03:42.921001+0000","last_fullsized":"2023-06-23T18:03:42.921001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:48:35.451788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.916514+0000","last_change":"2023-06-23T18:03:37.767363+0000","last_active":"2023-06-23T18:03:42.916514+0000","last_peered":"2023-06-23T18:03:42.916514+0000","last_clean":"2023-06-23T18:03:42.916514+0000","last_became_active":"2023-06-23T18:03:37.766918+0000","last_became_peered":"2023-06-23T18:03:37.766918+0000","last_unstale":"2023-06-23T18:03:42.916514+0000","last_undegraded":"2023-06-23T18:03:42.916514+0000","last_fullsized":"2023-06-23T18:03:42.916514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:08:31.121336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920725+0000","last_change":"2023-06-23T18:03:37.757526+0000","last_active":"2023-06-23T18:03:42.920725+0000","last_peered":"2023-06-23T18:03:42.920725+0000","last_clean":"2023-06-23T18:03:42.920725+0000","last_became_active":"2023-06-23T18:03:37.757223+0000","last_became_peered":"2023-06-23T18:03:37.757223+0000","last_unstale":"2023-06-23T18:03:42.920725+0000","last_undegraded":"2023-06-23T18:03:42.920725+0000","last_fullsized":"2023-06-23T18:03:42.920725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:58.557767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921071+0000","last_change":"2023-06-23T18:03:37.770932+0000","last_active":"2023-06-23T18:03:42.921071+0000","last_peered":"2023-06-23T18:03:42.921071+0000","last_clean":"2023-06-23T18:03:42.921071+0000","last_became_active":"2023-06-23T18:03:37.770633+0000","last_became_peered":"2023-06-23T18:03:37.770633+0000","last_unstale":"2023-06-23T18:03:42.921071+0000","last_undegraded":"2023-06-23T18:03:42.921071+0000","last_fullsized":"2023-06-23T18:03:42.921071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:50:13.840232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158738+0000","last_change":"2023-06-23T18:03:37.757510+0000","last_active":"2023-06-23T18:03:42.158738+0000","last_peered":"2023-06-23T18:03:42.158738+0000","last_clean":"2023-06-23T18:03:42.158738+0000","last_became_active":"2023-06-23T18:03:37.757203+0000","last_became_peered":"2023-06-23T18:03:37.757203+0000","last_unstale":"2023-06-23T18:03:42.158738+0000","last_undegraded":"2023-06-23T18:03:42.158738+0000","last_fullsized":"2023-06-23T18:03:42.158738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:25:37.629512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.153090+0000","last_change":"2023-06-23T18:03:37.760931+0000","last_active":"2023-06-23T18:03:42.153090+0000","last_peered":"2023-06-23T18:03:42.153090+0000","last_clean":"2023-06-23T18:03:42.153090+0000","last_became_active":"2023-06-23T18:03:37.760672+0000","last_became_peered":"2023-06-23T18:03:37.760672+0000","last_unstale":"2023-06-23T18:03:42.153090+0000","last_undegraded":"2023-06-23T18:03:42.153090+0000","last_fullsized":"2023-06-23T18:03:42.153090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:35.350379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158558+0000","last_change":"2023-06-23T18:03:37.761315+0000","last_active":"2023-06-23T18:03:42.158558+0000","last_peered":"2023-06-23T18:03:42.158558+0000","last_clean":"2023-06-23T18:03:42.158558+0000","last_became_active":"2023-06-23T18:03:37.760975+0000","last_became_peered":"2023-06-23T18:03:37.760975+0000","last_unstale":"2023-06-23T18:03:42.158558+0000","last_undegraded":"2023-06-23T18:03:42.158558+0000","last_fullsized":"2023-06-23T18:03:42.158558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:04.033803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922192+0000","last_change":"2023-06-23T18:03:37.754659+0000","last_active":"2023-06-23T18:03:42.922192+0000","last_peered":"2023-06-23T18:03:42.922192+0000","last_clean":"2023-06-23T18:03:42.922192+0000","last_became_active":"2023-06-23T18:03:37.751277+0000","last_became_peered":"2023-06-23T18:03:37.751277+0000","last_unstale":"2023-06-23T18:03:42.922192+0000","last_undegraded":"2023-06-23T18:03:42.922192+0000","last_fullsized":"2023-06-23T18:03:42.922192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.156785+0000","last_change":"2023-06-23T18:03:37.752494+0000","last_active":"2023-06-23T18:03:42.156785+0000","last_peered":"2023-06-23T18:03:42.156785+0000","last_clean":"2023-06-23T18:03:42.156785+0000","last_became_active":"2023-06-23T18:03:37.752149+0000","last_became_peered":"2023-06-23T18:03:37.752149+0000","last_unstale":"2023-06-23T18:03:42.156785+0000","last_undegraded":"2023-06-23T18:03:42.156785+0000","last_fullsized":"2023-06-23T18:03:42.156785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:20:37.654382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.921056+0000","last_change":"2023-06-23T18:03:37.775997+0000","last_active":"2023-06-23T18:03:42.921056+0000","last_peered":"2023-06-23T18:03:42.921056+0000","last_clean":"2023-06-23T18:03:42.921056+0000","last_became_active":"2023-06-23T18:03:37.775685+0000","last_became_peered":"2023-06-23T18:03:37.775685+0000","last_unstale":"2023-06-23T18:03:42.921056+0000","last_undegraded":"2023-06-23T18:03:42.921056+0000","last_fullsized":"2023-06-23T18:03:42.921056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:59:08.051946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923426+0000","last_change":"2023-06-23T18:03:37.763367+0000","last_active":"2023-06-23T18:03:42.923426+0000","last_peered":"2023-06-23T18:03:42.923426+0000","last_clean":"2023-06-23T18:03:42.923426+0000","last_became_active":"2023-06-23T18:03:37.763035+0000","last_became_peered":"2023-06-23T18:03:37.763035+0000","last_unstale":"2023-06-23T18:03:42.923426+0000","last_undegraded":"2023-06-23T18:03:42.923426+0000","last_fullsized":"2023-06-23T18:03:42.923426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:58.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152175+0000","last_change":"2023-06-23T18:03:37.758554+0000","last_active":"2023-06-23T18:03:42.152175+0000","last_peered":"2023-06-23T18:03:42.152175+0000","last_clean":"2023-06-23T18:03:42.152175+0000","last_became_active":"2023-06-23T18:03:37.758307+0000","last_became_peered":"2023-06-23T18:03:37.758307+0000","last_unstale":"2023-06-23T18:03:42.152175+0000","last_undegraded":"2023-06-23T18:03:42.152175+0000","last_fullsized":"2023-06-23T18:03:42.152175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:23:09.633003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152731+0000","last_change":"2023-06-23T18:03:37.760705+0000","last_active":"2023-06-23T18:03:42.152731+0000","last_peered":"2023-06-23T18:03:42.152731+0000","last_clean":"2023-06-23T18:03:42.152731+0000","last_became_active":"2023-06-23T18:03:37.759846+0000","last_became_peered":"2023-06-23T18:03:37.759846+0000","last_unstale":"2023-06-23T18:03:42.152731+0000","last_undegraded":"2023-06-23T18:03:42.152731+0000","last_fullsized":"2023-06-23T18:03:42.152731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:26.801591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158644+0000","last_change":"2023-06-23T18:03:37.757927+0000","last_active":"2023-06-23T18:03:42.158644+0000","last_peered":"2023-06-23T18:03:42.158644+0000","last_clean":"2023-06-23T18:03:42.158644+0000","last_became_active":"2023-06-23T18:03:37.757635+0000","last_became_peered":"2023-06-23T18:03:37.757635+0000","last_unstale":"2023-06-23T18:03:42.158644+0000","last_undegraded":"2023-06-23T18:03:42.158644+0000","last_fullsized":"2023-06-23T18:03:42.158644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:26.427732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.923450+0000","last_change":"2023-06-23T18:03:37.768282+0000","last_active":"2023-06-23T18:03:42.923450+0000","last_peered":"2023-06-23T18:03:42.923450+0000","last_clean":"2023-06-23T18:03:42.923450+0000","last_became_active":"2023-06-23T18:03:37.768038+0000","last_became_peered":"2023-06-23T18:03:37.768038+0000","last_unstale":"2023-06-23T18:03:42.923450+0000","last_undegraded":"2023-06-23T18:03:42.923450+0000","last_fullsized":"2023-06-23T18:03:42.923450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:03:37.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922018+0000","last_change":"2023-06-23T18:03:37.752981+0000","last_active":"2023-06-23T18:03:42.922018+0000","last_peered":"2023-06-23T18:03:42.922018+0000","last_clean":"2023-06-23T18:03:42.922018+0000","last_became_active":"2023-06-23T18:03:37.752291+0000","last_became_peered":"2023-06-23T18:03:37.752291+0000","last_unstale":"2023-06-23T18:03:42.922018+0000","last_undegraded":"2023-06-23T18:03:42.922018+0000","last_fullsized":"2023-06-23T18:03:42.922018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:14:05.699645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919440+0000","last_change":"2023-06-23T18:03:37.756558+0000","last_active":"2023-06-23T18:03:42.919440+0000","last_peered":"2023-06-23T18:03:42.919440+0000","last_clean":"2023-06-23T18:03:42.919440+0000","last_became_active":"2023-06-23T18:03:37.756044+0000","last_became_peered":"2023-06-23T18:03:37.756044+0000","last_unstale":"2023-06-23T18:03:42.919440+0000","last_undegraded":"2023-06-23T18:03:42.919440+0000","last_fullsized":"2023-06-23T18:03:42.919440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:17:54.349038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920901+0000","last_change":"2023-06-23T18:03:37.775992+0000","last_active":"2023-06-23T18:03:42.920901+0000","last_peered":"2023-06-23T18:03:42.920901+0000","last_clean":"2023-06-23T18:03:42.920901+0000","last_became_active":"2023-06-23T18:03:37.775667+0000","last_became_peered":"2023-06-23T18:03:37.775667+0000","last_unstale":"2023-06-23T18:03:42.920901+0000","last_undegraded":"2023-06-23T18:03:42.920901+0000","last_fullsized":"2023-06-23T18:03:42.920901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.157021+0000","last_change":"2023-06-23T18:03:37.752811+0000","last_active":"2023-06-23T18:03:42.157021+0000","last_peered":"2023-06-23T18:03:42.157021+0000","last_clean":"2023-06-23T18:03:42.157021+0000","last_became_active":"2023-06-23T18:03:37.752518+0000","last_became_peered":"2023-06-23T18:03:37.752518+0000","last_unstale":"2023-06-23T18:03:42.157021+0000","last_undegraded":"2023-06-23T18:03:42.157021+0000","last_fullsized":"2023-06-23T18:03:42.157021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:26.870651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.920914+0000","last_change":"2023-06-23T18:03:37.757769+0000","last_active":"2023-06-23T18:03:42.920914+0000","last_peered":"2023-06-23T18:03:42.920914+0000","last_clean":"2023-06-23T18:03:42.920914+0000","last_became_active":"2023-06-23T18:03:37.757520+0000","last_became_peered":"2023-06-23T18:03:37.757520+0000","last_unstale":"2023-06-23T18:03:42.920914+0000","last_undegraded":"2023-06-23T18:03:42.920914+0000","last_fullsized":"2023-06-23T18:03:42.920914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:39:30.351590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.919630+0000","last_change":"2023-06-23T18:03:37.755865+0000","last_active":"2023-06-23T18:03:42.919630+0000","last_peered":"2023-06-23T18:03:42.919630+0000","last_clean":"2023-06-23T18:03:42.919630+0000","last_became_active":"2023-06-23T18:03:37.755153+0000","last_became_peered":"2023-06-23T18:03:37.755153+0000","last_unstale":"2023-06-23T18:03:42.919630+0000","last_undegraded":"2023-06-23T18:03:42.919630+0000","last_fullsized":"2023-06-23T18:03:42.919630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:42:12.603650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.158475+0000","last_change":"2023-06-23T18:03:37.754864+0000","last_active":"2023-06-23T18:03:42.158475+0000","last_peered":"2023-06-23T18:03:42.158475+0000","last_clean":"2023-06-23T18:03:42.158475+0000","last_became_active":"2023-06-23T18:03:37.754386+0000","last_became_peered":"2023-06-23T18:03:37.754386+0000","last_unstale":"2023-06-23T18:03:42.158475+0000","last_undegraded":"2023-06-23T18:03:42.158475+0000","last_fullsized":"2023-06-23T18:03:42.158475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:30:06.917401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.922699+0000","last_change":"2023-06-23T18:03:37.757620+0000","last_active":"2023-06-23T18:03:42.922699+0000","last_peered":"2023-06-23T18:03:42.922699+0000","last_clean":"2023-06-23T18:03:42.922699+0000","last_became_active":"2023-06-23T18:03:37.757346+0000","last_became_peered":"2023-06-23T18:03:37.757346+0000","last_unstale":"2023-06-23T18:03:42.922699+0000","last_undegraded":"2023-06-23T18:03:42.922699+0000","last_fullsized":"2023-06-23T18:03:42.922699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:31:59.832808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-23T18:03:42.152893+0000","last_change":"2023-06-23T18:03:37.759638+0000","last_active":"2023-06-23T18:03:42.152893+0000","last_peered":"2023-06-23T18:03:42.152893+0000","last_clean":"2023-06-23T18:03:42.152893+0000","last_became_active":"2023-06-23T18:03:37.759380+0000","last_became_peered":"2023-06-23T18:03:37.759380+0000","last_unstale":"2023-06-23T18:03:42.152893+0000","last_undegraded":"2023-06-23T18:03:42.152893+0000","last_fullsized":"2023-06-23T18:03:42.152893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:37.339856+0000","last_clean_scrub_stamp":"2023-06-23T18:03:37.339856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:44.739308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4591,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31300,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340540,"statfs":{"total":96636764160,"available":96604712960,"internally_reserved":0,"allocated":258048,"data_stored":104812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":253952,"data_stored":104722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":851968,"data_stored":695672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:04:04.740 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-23T18:04:04.740 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-23T18:04:04.740 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-23T18:04:04.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-23T18:04:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 Processor -- start 2023-06-23T18:04:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- start start 2023-06-23T18:04:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc04415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440ca370 con 0x7fc04415c280 2023-06-23T18:04:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0440c8790 con 0x7fc04415beb0 2023-06-23T18:04:04.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc044160780 con 0x7fc04406d110 2023-06-23T18:04:04.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc041d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc04415c280 0x7fc0440c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37190/0 (socket says 172.21.15.22:37190) 2023-06-23T18:04:04.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc041d9b700 1 -- 172.21.15.22:0/784226055 learned_addr learned my addr 172.21.15.22:0/784226055 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:04.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc042d9d700 1 --2- 172.21.15.22:0/784226055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc04415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715359766 0 0) 0x7fc0440ca370 con 0x7fc04415c280 2023-06-23T18:04:04.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0300035d0 con 0x7fc04415c280 2023-06-23T18:04:04.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335859448 0 0) 0x7fc0440c8790 con 0x7fc04415beb0 2023-06-23T18:04:04.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0440ca370 con 0x7fc04415beb0 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc042d9d700 1 -- 172.21.15.22:0/784226055 >> v1:172.21.15.169:6789/0 conn(0x7fc04415beb0 legacy=0x7fc044070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc042d9d700 1 -- 172.21.15.22:0/784226055 >> v1:172.21.15.22:6789/0 conn(0x7fc04415c280 legacy=0x7fc0440c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc042d9d700 1 -- 172.21.15.22:0/784226055 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044071920 con 0x7fc04406d110 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc042d9d700 1 --2- 172.21.15.22:0/784226055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc04415fc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009880 tx=0x7fc03c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ddafc5db07d7af server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011830 con 0x7fc04406d110 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c011990 con 0x7fc04406d110 2023-06-23T18:04:04.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04159a700 1 -- 172.21.15.22:0/784226055 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011d00 con 0x7fc04406d110 2023-06-23T18:04:04.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- 172.21.15.22:0/784226055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 msgr2=0x7fc04415fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 --2- 172.21.15.22:0/784226055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc04415fc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009880 tx=0x7fc03c009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:04.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.831+0000 7fc04359e700 1 -- 172.21.15.22:0/784226055 shutdown_connections 2023-06-23T18:04:04.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 --2- 172.21.15.22:0/784226055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc04415fc00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 -- 172.21.15.22:0/784226055 >> 172.21.15.22:0/784226055 conn(0x7fc04406a5f0 msgr2=0x7fc0440caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 -- 172.21.15.22:0/784226055 shutdown_connections 2023-06-23T18:04:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 -- 172.21.15.22:0/784226055 wait complete. 2023-06-23T18:04:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 Processor -- start 2023-06-23T18:04:04.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.835+0000 7fc04359e700 1 -- start start 2023-06-23T18:04:04.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc0440c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04415c280 0x7fc04406bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b0e40 con 0x7fc04415beb0 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b1090 con 0x7fc04415c280 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b1370 con 0x7fc04406d110 2023-06-23T18:04:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44176/0 (socket says 172.21.15.22:44176) 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 -- 172.21.15.22:0/213939230 learned_addr learned my addr 172.21.15.22:0/213939230 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc042d9d700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04415c280 0x7fc04406bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04259c700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc0440c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 -- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 msgr2=0x7fc0440c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc0440c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 -- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04415c280 msgr2=0x7fc04406bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:04.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04415c280 0x7fc04406bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:04.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 -- 172.21.15.22:0/213939230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0094b0 con 0x7fc04415beb0 2023-06-23T18:04:04.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc041d9b700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c560 tx=0x7fc02c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c005d90 con 0x7fc04415beb0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c0070d0 con 0x7fc04415beb0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440b1650 con 0x7fc04415beb0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c007460 con 0x7fc04415beb0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0440c7f40 con 0x7fc04415beb0 2023-06-23T18:04:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.839+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440b1b70 con 0x7fc04415beb0 2023-06-23T18:04:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.843+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc02c0075c0 con 0x7fc04415beb0 2023-06-23T18:04:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.843+0000 7fc03affd700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 0x7fc0240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.843+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc02c0835f0 con 0x7fc04415beb0 2023-06-23T18:04:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.843+0000 7fc04259c700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 0x7fc0240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.843+0000 7fc04259c700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 0x7fc0240687a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc034001560 tx=0x7fc034009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:04.855+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc0440b1b70 con 0x7fc04415beb0 2023-06-23T18:04:05.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc0440b1b70 con 0x7fc04415beb0 2023-06-23T18:04:05.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc03affd700 1 -- 172.21.15.22:0/213939230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc0440b1b70 con 0x7fc04415beb0 2023-06-23T18:04:05.068 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:05.068 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-23T18:04:05.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 msgr2=0x7fc0240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 0x7fc0240687a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc034001560 tx=0x7fc034009040 comp rx=0 tx=0).stop 2023-06-23T18:04:05.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 msgr2=0x7fc0440c1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.051+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c560 tx=0x7fc02c00c870 comp rx=0 tx=0).stop 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 shutdown_connections 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc0240662f0 0x7fc0240687a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04415c280 0x7fc04406bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc04415beb0 0x7fc0440c1c20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 --2- 172.21.15.22:0/213939230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc04406d110 0x7fc0440c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 >> 172.21.15.22:0/213939230 conn(0x7fc04406a5f0 msgr2=0x7fc0440ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 shutdown_connections 2023-06-23T18:04:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.055+0000 7fc04359e700 1 -- 172.21.15.22:0/213939230 wait complete. 2023-06-23T18:04:05.081 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-23T18:04:05.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:05.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 Processor -- start 2023-06-23T18:04:05.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 -- start start 2023-06-23T18:04:05.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f281c06c610 0x7f281c0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0c9b50 con 0x7f281c070280 2023-06-23T18:04:05.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0c9e30 con 0x7f281c0b01a0 2023-06-23T18:04:05.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28229c0700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c1590a0 con 0x7f281c06c610 2023-06-23T18:04:05.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28221bf700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f281c06c610 0x7f281c0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28219be700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f281c070280 0x7f281c0afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37200/0 (socket says 172.21.15.22:37200) 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28219be700 1 -- 172.21.15.22:0/2078900453 learned_addr learned my addr 172.21.15.22:0/2078900453 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28221bf700 1 -- 172.21.15.22:0/2078900453 >> v1:172.21.15.169:6789/0 conn(0x7f281c0b01a0 legacy=0x7f281c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28209bc700 1 -- 172.21.15.22:0/2078900453 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988501785 0 0) 0x7f281c0c9b50 con 0x7f281c070280 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28221bf700 1 -- 172.21.15.22:0/2078900453 >> v1:172.21.15.22:6789/0 conn(0x7f281c070280 legacy=0x7f281c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.175+0000 7f28221bf700 1 -- 172.21.15.22:0/2078900453 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0ca110 con 0x7f281c06c610 2023-06-23T18:04:05.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28221bf700 1 --2- 172.21.15.22:0/2078900453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c0c9570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2814009850 tx=0x7f2814009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aad494e22c6b3f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28209bc700 1 -- 172.21.15.22:0/2078900453 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011830 con 0x7f281c06c610 2023-06-23T18:04:05.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28209bc700 1 -- 172.21.15.22:0/2078900453 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2814011990 con 0x7f281c06c610 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28209bc700 1 -- 172.21.15.22:0/2078900453 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011d20 con 0x7f281c06c610 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 -- 172.21.15.22:0/2078900453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 msgr2=0x7f281c0c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 --2- 172.21.15.22:0/2078900453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c0c9570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2814009850 tx=0x7f2814009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 -- 172.21.15.22:0/2078900453 shutdown_connections 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 --2- 172.21.15.22:0/2078900453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c0c9570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 -- 172.21.15.22:0/2078900453 >> 172.21.15.22:0/2078900453 conn(0x7f281c06a5f0 msgr2=0x7f281c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 -- 172.21.15.22:0/2078900453 shutdown_connections 2023-06-23T18:04:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.179+0000 7f28229c0700 1 -- 172.21.15.22:0/2078900453 wait complete. 2023-06-23T18:04:05.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 Processor -- start 2023-06-23T18:04:05.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- start start 2023-06-23T18:04:05.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 0x7f281c14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 0x7f281c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1584c0 con 0x7f281c06c610 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c157750 con 0x7f281c070280 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0ae8c0 con 0x7f281c0b01a0 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44180/0 (socket says 172.21.15.22:44180) 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 -- 172.21.15.22:0/2863990640 learned_addr learned my addr 172.21.15.22:0/2863990640 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28221bf700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 0x7f281c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28211bd700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 0x7f281c14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 -- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 msgr2=0x7f281c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 0x7f281c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 -- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 msgr2=0x7f281c14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 0x7f281c14a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 -- 172.21.15.22:0/2863990640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140094b0 con 0x7f281c06c610 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28221bf700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 0x7f281c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:05.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28211bd700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 0x7f281c14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28219be700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c550 tx=0x7f280c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c004ca0 con 0x7f281c06c610 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c004e00 con 0x7f281c06c610 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c0aeba0 con 0x7f281c06c610 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c007310 con 0x7f281c06c610 2023-06-23T18:04:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.183+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c0af160 con 0x7f281c06c610 2023-06-23T18:04:05.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c04e4c0 con 0x7f281c06c610 2023-06-23T18:04:05.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f280c007470 con 0x7f281c06c610 2023-06-23T18:04:05.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28127fc700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 0x7f28080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f280c083d50 con 0x7f281c06c610 2023-06-23T18:04:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28211bd700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 0x7f28080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.187+0000 7f28211bd700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 0x7f28080687a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f281c15c440 tx=0x7f2818009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.191+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f280c087030 con 0x7f281c06c610 2023-06-23T18:04:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.351+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f281c0b0780 con 0x7f281c06c610 2023-06-23T18:04:05.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.355+0000 7f28127fc700 1 -- 172.21.15.22:0/2863990640 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f280c050ee0 con 0x7f281c06c610 2023-06-23T18:04:05.372 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:05.372 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 msgr2=0x7f28080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 0x7f28080687a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f281c15c440 tx=0x7f2818009040 comp rx=0 tx=0).stop 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 msgr2=0x7f281c14e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c550 tx=0x7f280c00c910 comp rx=0 tx=0).stop 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 shutdown_connections 2023-06-23T18:04:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f281c0b01a0 0x7f281c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f28080662f0 0x7f28080687a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f281c070280 0x7f281c14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 --2- 172.21.15.22:0/2863990640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281c06c610 0x7f281c14e090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 >> 172.21.15.22:0/2863990640 conn(0x7f281c06a5f0 msgr2=0x7f281c0ab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 shutdown_connections 2023-06-23T18:04:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.359+0000 7f28229c0700 1 -- 172.21.15.22:0/2863990640 wait complete. 2023-06-23T18:04:05.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:05.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:05.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 Processor -- start 2023-06-23T18:04:05.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 -- start start 2023-06-23T18:04:05.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580cf3a0 0x7fdd580cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdd580d1f30 0x7fdd580d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580ccb30 con 0x7fdd580d2360 2023-06-23T18:04:05.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580cfcb0 con 0x7fdd580d1f30 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.491+0000 7fdd5d47e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580d2880 con 0x7fdd580cf3a0 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5cc7d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdd580d2360 0x7fdd5815f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37216/0 (socket says 172.21.15.22:37216) 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5cc7d700 1 -- 172.21.15.22:0/2526215416 learned_addr learned my addr 172.21.15.22:0/2526215416 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 --2- 172.21.15.22:0/2526215416 >> v2:172.21.15.22:3300/0 conn(0x7fdd580d1f30 0x7fdd580d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd57fff700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580cf3a0 0x7fdd580cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543299367 0 0) 0x7fdd580ccb30 con 0x7fdd580d2360 2023-06-23T18:04:05.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd440035d0 con 0x7fdd580d2360 2023-06-23T18:04:05.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083966517 0 0) 0x7fdd440035d0 con 0x7fdd580d2360 2023-06-23T18:04:05.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 -- 172.21.15.22:0/2526215416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580cf3a0 msgr2=0x7fdd580cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:05.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580cf3a0 0x7fdd580cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 -- 172.21.15.22:0/2526215416 >> v1:172.21.15.22:6789/0 conn(0x7fdd580d2360 legacy=0x7fdd5815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 -- 172.21.15.22:0/2526215416 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580d2b60 con 0x7fdd580d1f30 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd577fe700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580d1f30 0x7fdd580d0020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009850 tx=0x7fdd40009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab541f01923177dd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500019f0 con 0x7fdd580d1f30 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40004030 con 0x7fdd580d1f30 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd56ffd700 1 -- 172.21.15.22:0/2526215416 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400043a0 con 0x7fdd580d1f30 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 -- 172.21.15.22:0/2526215416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580d1f30 msgr2=0x7fdd580d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580d1f30 0x7fdd580d0020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009850 tx=0x7fdd40009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:05.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 -- 172.21.15.22:0/2526215416 shutdown_connections 2023-06-23T18:04:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580d1f30 0x7fdd580d0020 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/2526215416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580cf3a0 0x7fdd580cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 -- 172.21.15.22:0/2526215416 >> 172.21.15.22:0/2526215416 conn(0x7fdd5806a280 msgr2=0x7fdd5809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 -- 172.21.15.22:0/2526215416 shutdown_connections 2023-06-23T18:04:05.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.495+0000 7fdd5d47e700 1 -- 172.21.15.22:0/2526215416 wait complete. 2023-06-23T18:04:05.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 Processor -- start 2023-06-23T18:04:05.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 -- start start 2023-06-23T18:04:05.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580d1f30 0x7fdd580b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580d2360 0x7fdd580b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58160300 con 0x7fdd580cf3a0 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580b6870 con 0x7fdd580d1f30 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580b6b50 con 0x7fdd580d2360 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44192/0 (socket says 172.21.15.22:44192) 2023-06-23T18:04:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 -- 172.21.15.22:0/1685101814 learned_addr learned my addr 172.21.15.22:0/1685101814 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5cc7d700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580d2360 0x7fdd580b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd577fe700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580d1f30 0x7fdd580b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 -- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580d2360 msgr2=0x7fdd580b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580d2360 0x7fdd580b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 -- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580d1f30 msgr2=0x7fdd580b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580d1f30 0x7fdd580b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 -- 172.21.15.22:0/1685101814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400094b0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd57fff700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800e9d0 tx=0x7fdd4800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48009730 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800ee70 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd580b6e30 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.499+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800ca70 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5815c350 con 0x7fdd580cf3a0 2023-06-23T18:04:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5815c8e0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdd4800cbd0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd54ff9700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 0x7fdd380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd48084460 con 0x7fdd580cf3a0 2023-06-23T18:04:05.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.503+0000 7fdd577fe700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 0x7fdd380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.507+0000 7fdd577fe700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 0x7fdd380687a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd400090b0 tx=0x7fdd40008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.511+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fdd5815c8e0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.675+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd5815c8e0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.675+0000 7fdd54ff9700 1 -- 172.21.15.22:0/1685101814 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd5815c8e0 con 0x7fdd580cf3a0 2023-06-23T18:04:05.691 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 msgr2=0x7fdd380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 0x7fdd380687a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd400090b0 tx=0x7fdd40008030 comp rx=0 tx=0).stop 2023-06-23T18:04:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 msgr2=0x7fdd58156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800e9d0 tx=0x7fdd4800ece0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 shutdown_connections 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd580d2360 0x7fdd580b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdd380662f0 0x7fdd380687a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580d1f30 0x7fdd580b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 --2- 172.21.15.22:0/1685101814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd580cf3a0 0x7fdd58156b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 >> 172.21.15.22:0/1685101814 conn(0x7fdd5806a280 msgr2=0x7fdd580cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 shutdown_connections 2023-06-23T18:04:05.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.679+0000 7fdd5d47e700 1 -- 172.21.15.22:0/1685101814 wait complete. 2023-06-23T18:04:05.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 Processor -- start 2023-06-23T18:04:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- start start 2023-06-23T18:04:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 0x7f35840d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35840ccb30 con 0x7f35840d2360 2023-06-23T18:04:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35840d2880 con 0x7f35840cf3a0 2023-06-23T18:04:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35840d2b60 con 0x7f35840d1f30 2023-06-23T18:04:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358359e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f35840d2360 0x7f35840ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37230/0 (socket says 172.21.15.22:37230) 2023-06-23T18:04:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358359e700 1 -- 172.21.15.22:0/3499139834 learned_addr learned my addr 172.21.15.22:0/3499139834 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f357bfff700 1 --2- 172.21.15.22:0/3499139834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 0x7f35840d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358259c700 1 -- 172.21.15.22:0/3499139834 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269740487 0 0) 0x7f35840ccb30 con 0x7f35840d2360 2023-06-23T18:04:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358259c700 1 -- 172.21.15.22:0/3499139834 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35700035d0 con 0x7f35840d2360 2023-06-23T18:04:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f357bfff700 1 -- 172.21.15.22:0/3499139834 >> v1:172.21.15.169:6789/0 conn(0x7f35840cf3a0 legacy=0x7f35840cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f357bfff700 1 -- 172.21.15.22:0/3499139834 >> v1:172.21.15.22:6789/0 conn(0x7f35840d2360 legacy=0x7f35840ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f357bfff700 1 -- 172.21.15.22:0/3499139834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35840d2e40 con 0x7f35840d1f30 2023-06-23T18:04:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f357bfff700 1 --2- 172.21.15.22:0/3499139834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 0x7f35840d0020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f356c009880 tx=0x7f356c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b837868daa03fe1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358259c700 1 -- 172.21.15.22:0/3499139834 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c011830 con 0x7f35840d1f30 2023-06-23T18:04:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358259c700 1 -- 172.21.15.22:0/3499139834 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c011990 con 0x7f35840d1f30 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f358259c700 1 -- 172.21.15.22:0/3499139834 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c011d20 con 0x7f35840d1f30 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- 172.21.15.22:0/3499139834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 msgr2=0x7f35840d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 --2- 172.21.15.22:0/3499139834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 0x7f35840d0020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f356c009880 tx=0x7f356c009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- 172.21.15.22:0/3499139834 shutdown_connections 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 --2- 172.21.15.22:0/3499139834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840d1f30 0x7f35840d0020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- 172.21.15.22:0/3499139834 >> 172.21.15.22:0/3499139834 conn(0x7f358406a280 msgr2=0x7f358409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- 172.21.15.22:0/3499139834 shutdown_connections 2023-06-23T18:04:05.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.803+0000 7f35888c4700 1 -- 172.21.15.22:0/3499139834 wait complete. 2023-06-23T18:04:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 Processor -- start 2023-06-23T18:04:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 -- start start 2023-06-23T18:04:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 0x7f35840b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 0x7f358415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35840d2360 0x7f35840c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358415d4b0 con 0x7f35840d1f30 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358415c930 con 0x7f35840d2360 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358415cc10 con 0x7f35840cf3a0 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f357bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 0x7f358415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f357bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 0x7f358415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44200/0 (socket says 172.21.15.22:44200) 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f357bfff700 1 -- 172.21.15.22:0/255533075 learned_addr learned my addr 172.21.15.22:0/255533075 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 0x7f35840b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f358359e700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35840d2360 0x7f35840c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 -- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35840d2360 msgr2=0x7f35840c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35840d2360 0x7f35840c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 -- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 msgr2=0x7f358415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 0x7f358415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 -- 172.21.15.22:0/255533075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0094b0 con 0x7f35840cf3a0 2023-06-23T18:04:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f3582d9d700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 0x7f35840b7d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f357400c430 tx=0x7f357400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740052d0 con 0x7f35840cf3a0 2023-06-23T18:04:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3574005450 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.807+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358415cef0 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574006b10 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358415c620 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358415d720 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f35740036f0 con 0x7f35840cf3a0 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357b7fe700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 0x7f35640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357bfff700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 0x7f35640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3574083130 con 0x7f35840cf3a0 2023-06-23T18:04:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.811+0000 7f357bfff700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 0x7f35640687a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f356c0097d0 tx=0x7f356c010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:05.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.819+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f358415d720 con 0x7f35840cf3a0 2023-06-23T18:04:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.987+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f358415d720 con 0x7f35840cf3a0 2023-06-23T18:04:06.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.987+0000 7f357b7fe700 1 -- 172.21.15.22:0/255533075 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35740502a0 con 0x7f35840cf3a0 2023-06-23T18:04:06.004 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:06.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 msgr2=0x7f35640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 0x7f35640687a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f356c0097d0 tx=0x7f356c010000 comp rx=0 tx=0).stop 2023-06-23T18:04:06.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 msgr2=0x7f35840b7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 0x7f35840b7d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f357400c430 tx=0x7f357400c740 comp rx=0 tx=0).stop 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 shutdown_connections 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f35640662f0 0x7f35640687a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35840d2360 0x7f35840c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35840d1f30 0x7f358415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 --2- 172.21.15.22:0/255533075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35840cf3a0 0x7f35840b7d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 >> 172.21.15.22:0/255533075 conn(0x7f358406a280 msgr2=0x7f35840b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 shutdown_connections 2023-06-23T18:04:06.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:05.991+0000 7f35888c4700 1 -- 172.21.15.22:0/255533075 wait complete. 2023-06-23T18:04:06.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:06.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 Processor -- start 2023-06-23T18:04:06.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 -- start start 2023-06-23T18:04:06.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f749c00c410 0x7f749c0530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0554f0 con 0x7f749c039d50 2023-06-23T18:04:06.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c051560 con 0x7f749c00e580 2023-06-23T18:04:06.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aabd2700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c051840 con 0x7f749c00c410 2023-06-23T18:04:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aa3d1700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f749c039d50 0x7f749c055cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37244/0 (socket says 172.21.15.22:37244) 2023-06-23T18:04:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a9bd0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f749c00c410 0x7f749c0530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74aa3d1700 1 -- 172.21.15.22:0/2088608196 learned_addr learned my addr 172.21.15.22:0/2088608196 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a9bd0700 1 -- 172.21.15.22:0/2088608196 >> v1:172.21.15.169:6789/0 conn(0x7f749c00e580 legacy=0x7f749c00c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a9bd0700 1 -- 172.21.15.22:0/2088608196 >> v1:172.21.15.22:6789/0 conn(0x7f749c039d50 legacy=0x7f749c055cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a9bd0700 1 -- 172.21.15.22:0/2088608196 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c055790 con 0x7f749c00c410 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a8bce700 1 -- 172.21.15.22:0/2088608196 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900834360 0 0) 0x7f749c0554f0 con 0x7f749c039d50 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a9bd0700 1 --2- 172.21.15.22:0/2088608196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c00c410 0x7f749c0530c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009850 tx=0x7f74a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320866b6c2fe0c85 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a8bce700 1 -- 172.21.15.22:0/2088608196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0011830 con 0x7f749c00c410 2023-06-23T18:04:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a8bce700 1 -- 172.21.15.22:0/2088608196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a0011990 con 0x7f749c00c410 2023-06-23T18:04:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.131+0000 7f74a8bce700 1 -- 172.21.15.22:0/2088608196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0011d20 con 0x7f749c00c410 2023-06-23T18:04:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 -- 172.21.15.22:0/2088608196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c00c410 msgr2=0x7f749c0530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 --2- 172.21.15.22:0/2088608196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c00c410 0x7f749c0530c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009850 tx=0x7f74a0009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 -- 172.21.15.22:0/2088608196 shutdown_connections 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 --2- 172.21.15.22:0/2088608196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c00c410 0x7f749c0530c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 -- 172.21.15.22:0/2088608196 >> 172.21.15.22:0/2088608196 conn(0x7f749c053980 msgr2=0x7f749c053d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 -- 172.21.15.22:0/2088608196 shutdown_connections 2023-06-23T18:04:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.135+0000 7f74aabd2700 1 -- 172.21.15.22:0/2088608196 wait complete. 2023-06-23T18:04:06.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 Processor -- start 2023-06-23T18:04:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 -- start start 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 0x7f749c050b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f749c00e580 0x7f749c04bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0510c0 con 0x7f749c039d50 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c04c9e0 con 0x7f749c00e580 2023-06-23T18:04:06.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aabd2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c04ccc0 con 0x7f749c00c410 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74a9bd0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 0x7f749c050b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44206/0 (socket says 172.21.15.22:44206) 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 -- 172.21.15.22:0/1263858251 learned_addr learned my addr 172.21.15.22:0/1263858251 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74a93cf700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f749c00e580 0x7f749c04bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 -- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 msgr2=0x7f749c050b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 0x7f749c050b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 -- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f749c00e580 msgr2=0x7f749c04bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f749c00e580 0x7f749c04bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 -- 172.21.15.22:0/1263858251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00094b0 con 0x7f749c039d50 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74a9bd0700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 0x7f749c050b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f74aa3d1700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74a404ef50 tx=0x7f74a404ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.139+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4069960 con 0x7f749c039d50 2023-06-23T18:04:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a4069ac0 con 0x7f749c039d50 2023-06-23T18:04:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c04cfa0 con 0x7f749c039d50 2023-06-23T18:04:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4069e50 con 0x7f749c039d50 2023-06-23T18:04:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c04d560 con 0x7f749c039d50 2023-06-23T18:04:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c051230 con 0x7f749c039d50 2023-06-23T18:04:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f74a4066880 con 0x7f749c039d50 2023-06-23T18:04:06.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f749a7fc700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 0x7f74880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f74a40fc6e0 con 0x7f749c039d50 2023-06-23T18:04:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.143+0000 7f74a9bd0700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 0x7f74880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.147+0000 7f74a9bd0700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 0x7f74880687a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74a00090b0 tx=0x7f74a0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.151+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f74a40c9860 con 0x7f749c039d50 2023-06-23T18:04:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.315+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f749c0059c0 con 0x7f749c039d50 2023-06-23T18:04:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.315+0000 7f749a7fc700 1 -- 172.21.15.22:0/1263858251 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74a40cce80 con 0x7f749c039d50 2023-06-23T18:04:06.332 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 msgr2=0x7f74880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 0x7f74880687a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74a00090b0 tx=0x7f74a0008030 comp rx=0 tx=0).stop 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 msgr2=0x7f749c0ddc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74a404ef50 tx=0x7f74a404ed30 comp rx=0 tx=0).stop 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 shutdown_connections 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f749c039d50 0x7f749c0ddc20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74880662f0 0x7f74880687a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f749c00e580 0x7f749c04bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 --2- 172.21.15.22:0/1263858251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f749c00c410 0x7f749c050b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 >> 172.21.15.22:0/1263858251 conn(0x7f749c053980 msgr2=0x7f749c053d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 shutdown_connections 2023-06-23T18:04:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.319+0000 7f74aabd2700 1 -- 172.21.15.22:0/1263858251 wait complete. 2023-06-23T18:04:06.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:06.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.455+0000 7f42a6c60700 1 Processor -- start 2023-06-23T18:04:06.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.455+0000 7f42a6c60700 1 -- start start 2023-06-23T18:04:06.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.455+0000 7f42a6c60700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42a0158ef0 0x7f42a006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a006eab0 con 0x7f42a0158b20 2023-06-23T18:04:06.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a00cd680 con 0x7f42a0070e50 2023-06-23T18:04:06.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a00cdf20 con 0x7f42a0158ef0 2023-06-23T18:04:06.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a5c5e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f42a0158b20 0x7f42a0070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57326/0 (socket says 172.21.15.22:57326) 2023-06-23T18:04:06.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a5c5e700 1 -- 172.21.15.22:0/3749327497 learned_addr learned my addr 172.21.15.22:0/3749327497 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:06.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a545d700 1 --2- 172.21.15.22:0/3749327497 >> v2:172.21.15.169:3300/0 conn(0x7f42a0158ef0 0x7f42a006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a4c5c700 1 -- 172.21.15.22:0/3749327497 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880774737 0 0) 0x7f42a006eab0 con 0x7f42a0158b20 2023-06-23T18:04:06.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a4c5c700 1 -- 172.21.15.22:0/3749327497 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c0035d0 con 0x7f42a0158b20 2023-06-23T18:04:06.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a545d700 1 -- 172.21.15.22:0/3749327497 >> v1:172.21.15.169:6789/0 conn(0x7f42a0070e50 legacy=0x7f42a00c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a545d700 1 -- 172.21.15.22:0/3749327497 >> v1:172.21.15.22:6789/0 conn(0x7f42a0158b20 legacy=0x7f42a0070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a545d700 1 -- 172.21.15.22:0/3749327497 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00cd960 con 0x7f42a0158ef0 2023-06-23T18:04:06.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a545d700 1 --2- 172.21.15.22:0/3749327497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a006e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4290009850 tx=0x7f4290009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a9306ef6b38434 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a4c5c700 1 -- 172.21.15.22:0/3749327497 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011830 con 0x7f42a0158ef0 2023-06-23T18:04:06.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a4c5c700 1 -- 172.21.15.22:0/3749327497 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290011990 con 0x7f42a0158ef0 2023-06-23T18:04:06.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a4c5c700 1 -- 172.21.15.22:0/3749327497 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011d40 con 0x7f42a0158ef0 2023-06-23T18:04:06.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- 172.21.15.22:0/3749327497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 msgr2=0x7f42a006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 --2- 172.21.15.22:0/3749327497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a006e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4290009850 tx=0x7f4290009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:06.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- 172.21.15.22:0/3749327497 shutdown_connections 2023-06-23T18:04:06.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 --2- 172.21.15.22:0/3749327497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a006e440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- 172.21.15.22:0/3749327497 >> 172.21.15.22:0/3749327497 conn(0x7f42a006b700 msgr2=0x7f42a00b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- 172.21.15.22:0/3749327497 shutdown_connections 2023-06-23T18:04:06.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.459+0000 7f42a6c60700 1 -- 172.21.15.22:0/3749327497 wait complete. 2023-06-23T18:04:06.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 Processor -- start 2023-06-23T18:04:06.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 -- start start 2023-06-23T18:04:06.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 0x7f42a0154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a01f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0154a40 con 0x7f42a0158b20 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a015b210 con 0x7f42a0158ef0 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.463+0000 7f42a6c60700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a015b4f0 con 0x7f42a0070e50 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50150/0 (socket says 172.21.15.22:50150) 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 -- 172.21.15.22:0/2830638168 learned_addr learned my addr 172.21.15.22:0/2830638168 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a645f700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a01f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a5c5e700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 0x7f42a0154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 -- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 msgr2=0x7f42a0154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 0x7f42a0154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 -- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 msgr2=0x7f42a01f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a01f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 -- 172.21.15.22:0/2830638168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900094b0 con 0x7f42a0158b20 2023-06-23T18:04:06.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a5c5e700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 0x7f42a0154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:06.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a545d700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4290014040 tx=0x7f4290004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290019a00 con 0x7f42a0158b20 2023-06-23T18:04:06.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429001e070 con 0x7f42a0158b20 2023-06-23T18:04:06.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290019da0 con 0x7f42a0158b20 2023-06-23T18:04:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a01e4ea0 con 0x7f42a0158b20 2023-06-23T18:04:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a015b7d0 con 0x7f42a0158b20 2023-06-23T18:04:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.467+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a004e4c0 con 0x7f42a0158b20 2023-06-23T18:04:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.471+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f429002d450 con 0x7f42a0158b20 2023-06-23T18:04:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.471+0000 7f42967fc700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 0x7f42840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.471+0000 7f42a5c5e700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 0x7f42840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.471+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4290085010 con 0x7f42a0158b20 2023-06-23T18:04:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.471+0000 7f42a5c5e700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 0x7f42840687a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f429c0011e0 tx=0x7f429c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.475+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f42900521b0 con 0x7f42a0158b20 2023-06-23T18:04:06.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.639+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42a006a590 con 0x7f42a0158b20 2023-06-23T18:04:06.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42967fc700 1 -- 172.21.15.22:0/2830638168 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42900200a0 con 0x7f42a0158b20 2023-06-23T18:04:06.657 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:06.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 msgr2=0x7f42840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 0x7f42840687a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f429c0011e0 tx=0x7f429c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:06.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 msgr2=0x7f42a01f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4290014040 tx=0x7f4290004770 comp rx=0 tx=0).stop 2023-06-23T18:04:06.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 shutdown_connections 2023-06-23T18:04:06.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f42840662f0 0x7f42840687a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a0158ef0 0x7f42a01f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42a0158b20 0x7f42a01f5040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 --2- 172.21.15.22:0/2830638168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a0070e50 0x7f42a0154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 >> 172.21.15.22:0/2830638168 conn(0x7f42a006b700 msgr2=0x7f42a0069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 shutdown_connections 2023-06-23T18:04:06.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.643+0000 7f42a6c60700 1 -- 172.21.15.22:0/2830638168 wait complete. 2023-06-23T18:04:06.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:06.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 Processor -- start 2023-06-23T18:04:06.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 -- start start 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5b80afaf0 0x7fd5b80afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5b815aec0 0x7fd5b815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b815aa30 con 0x7fd5b8159e40 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b815e790 con 0x7fd5b80afaf0 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5bce02700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b80b0400 con 0x7fd5b815aec0 2023-06-23T18:04:06.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5b80afaf0 0x7fd5b80afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5b80afaf0 0x7fd5b80afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50156/0 (socket says 172.21.15.22:50156) 2023-06-23T18:04:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 -- 172.21.15.22:0/2699311416 learned_addr learned my addr 172.21.15.22:0/2699311416 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5aeffd700 1 --2- 172.21.15.22:0/2699311416 >> v2:172.21.15.169:3300/0 conn(0x7fd5b815aec0 0x7fd5b815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 -- 172.21.15.22:0/2699311416 >> v2:172.21.15.169:3300/0 conn(0x7fd5b815aec0 msgr2=0x7fd5b815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 --2- 172.21.15.22:0/2699311416 >> v2:172.21.15.169:3300/0 conn(0x7fd5b815aec0 0x7fd5b815d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 -- 172.21.15.22:0/2699311416 >> v1:172.21.15.169:6789/0 conn(0x7fd5b8159e40 legacy=0x7fd5b815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 -- 172.21.15.22:0/2699311416 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b815e4b0 con 0x7fd5b80afaf0 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b77fe700 1 --2- 172.21.15.22:0/2699311416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b80afaf0 0x7fd5b80afec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5b806fd20 tx=0x7fd5a8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ed5004be8c14e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b6ffd700 1 -- 172.21.15.22:0/2699311416 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800b8a0 con 0x7fd5b80afaf0 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b6ffd700 1 -- 172.21.15.22:0/2699311416 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a800ba00 con 0x7fd5b80afaf0 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.779+0000 7fd5b6ffd700 1 -- 172.21.15.22:0/2699311416 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800bd90 con 0x7fd5b80afaf0 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- 172.21.15.22:0/2699311416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b80afaf0 msgr2=0x7fd5b80afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2699311416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b80afaf0 0x7fd5b80afec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5b806fd20 tx=0x7fd5a8009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- 172.21.15.22:0/2699311416 shutdown_connections 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2699311416 >> v2:172.21.15.169:3300/0 conn(0x7fd5b815aec0 0x7fd5b815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2699311416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b80afaf0 0x7fd5b80afec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- 172.21.15.22:0/2699311416 >> 172.21.15.22:0/2699311416 conn(0x7fd5b806a5f0 msgr2=0x7fd5b80caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- 172.21.15.22:0/2699311416 shutdown_connections 2023-06-23T18:04:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- 172.21.15.22:0/2699311416 wait complete. 2023-06-23T18:04:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 Processor -- start 2023-06-23T18:04:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- start start 2023-06-23T18:04:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b80afaf0 0x7fd5b8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5b8159e40 0x7fd5b80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80cae30 con 0x7fd5b815aec0 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80cd800 con 0x7fd5b80afaf0 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.783+0000 7fd5bce02700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80ca460 con 0x7fd5b8159e40 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50162/0 (socket says 172.21.15.22:50162) 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 -- 172.21.15.22:0/2365939034 learned_addr learned my addr 172.21.15.22:0/2365939034 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b77fe700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b80afaf0 0x7fd5b8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5aeffd700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5b8159e40 0x7fd5b80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 -- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5b8159e40 msgr2=0x7fd5b80ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5b8159e40 0x7fd5b80ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 -- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b80afaf0 msgr2=0x7fd5b8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b80afaf0 0x7fd5b8151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 -- 172.21.15.22:0/2365939034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8009570 con 0x7fd5b815aec0 2023-06-23T18:04:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b7fff700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000cc50 tx=0x7fd5b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003c20 con 0x7fd5b815aec0 2023-06-23T18:04:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0003d80 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b80cd520 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000aab0 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b80ca0f0 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.787+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b80ca610 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.791+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd5b000ac10 con 0x7fd5b815aec0 2023-06-23T18:04:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.791+0000 7fd5b4ff9700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 0x7fd5980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.791+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd5b0083fb0 con 0x7fd5b815aec0 2023-06-23T18:04:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.791+0000 7fd5b77fe700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 0x7fd5980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.791+0000 7fd5b77fe700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 0x7fd5980687a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8000c00 tx=0x7fd5a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.799+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd5b80ca610 con 0x7fd5b815aec0 2023-06-23T18:04:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.963+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd5b80ca280 con 0x7fd5b815aec0 2023-06-23T18:04:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.963+0000 7fd5b4ff9700 1 -- 172.21.15.22:0/2365939034 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5b0051130 con 0x7fd5b815aec0 2023-06-23T18:04:06.979 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 msgr2=0x7fd5980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 0x7fd5980687a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8000c00 tx=0x7fd5a8008030 comp rx=0 tx=0).stop 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 msgr2=0x7fd5b80cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000cc50 tx=0x7fd5b000cf60 comp rx=0 tx=0).stop 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 shutdown_connections 2023-06-23T18:04:06.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5b815aec0 0x7fd5b80cc650 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5b8159e40 0x7fd5b80ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd5980662f0 0x7fd5980687a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 --2- 172.21.15.22:0/2365939034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b80afaf0 0x7fd5b8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 >> 172.21.15.22:0/2365939034 conn(0x7fd5b806a5f0 msgr2=0x7fd5b80ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 shutdown_connections 2023-06-23T18:04:06.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:06.967+0000 7fd5bce02700 1 -- 172.21.15.22:0/2365939034 wait complete. 2023-06-23T18:04:06.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 Processor -- start 2023-06-23T18:04:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 -- start start 2023-06-23T18:04:07.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6d940cf480 0x7f6d940ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cc9e0 0x7f6d940d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940b7ec0 con 0x7f6d940b7af0 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d940b1ad0 con 0x7f6d940cf480 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d93fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940d0a10 con 0x7f6d940cc9e0 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6d940cf480 0x7f6d940ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6d940cf480 0x7f6d940ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50166/0 (socket says 172.21.15.22:50166) 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 -- 172.21.15.22:0/2766605734 learned_addr learned my addr 172.21.15.22:0/2766605734 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d937fe700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cc9e0 0x7f6d940d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 -- 172.21.15.22:0/2766605734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cc9e0 msgr2=0x7f6d940d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cc9e0 0x7f6d940d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 -- 172.21.15.22:0/2766605734 >> v1:172.21.15.169:6789/0 conn(0x7f6d940b7af0 legacy=0x7f6d940ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 -- 172.21.15.22:0/2766605734 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940d0cb0 con 0x7f6d940cf480 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.095+0000 7f6d927fc700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cf480 0x7f6d940ce540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009e50 tx=0x7f6d8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608d5c3dbf11cd8f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d91ffb700 1 -- 172.21.15.22:0/2766605734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800e070 con 0x7f6d940cf480 2023-06-23T18:04:07.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d91ffb700 1 -- 172.21.15.22:0/2766605734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800b910 con 0x7f6d940cf480 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d91ffb700 1 -- 172.21.15.22:0/2766605734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800bcc0 con 0x7f6d940cf480 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- 172.21.15.22:0/2766605734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cf480 msgr2=0x7f6d940ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cf480 0x7f6d940ce540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009e50 tx=0x7f6d8800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- 172.21.15.22:0/2766605734 shutdown_connections 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cc9e0 0x7f6d940d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 --2- 172.21.15.22:0/2766605734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cf480 0x7f6d940ce540 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- 172.21.15.22:0/2766605734 >> 172.21.15.22:0/2766605734 conn(0x7f6d94069960 msgr2=0x7f6d94069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- 172.21.15.22:0/2766605734 shutdown_connections 2023-06-23T18:04:07.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- 172.21.15.22:0/2766605734 wait complete. 2023-06-23T18:04:07.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 Processor -- start 2023-06-23T18:04:07.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.099+0000 7f6d93fff700 1 -- start start 2023-06-23T18:04:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 0x7f6d940c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cf480 0x7f6d941557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c7070 con 0x7f6d940cc9e0 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9415d900 con 0x7f6d940b7af0 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c6480 con 0x7f6d940cf480 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50170/0 (socket says 172.21.15.22:50170) 2023-06-23T18:04:07.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 -- 172.21.15.22:0/3156034682 learned_addr learned my addr 172.21.15.22:0/3156034682 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d937fe700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cf480 0x7f6d941557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d92ffd700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 0x7f6d940c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 -- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cf480 msgr2=0x7f6d941557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cf480 0x7f6d941557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 -- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 msgr2=0x7f6d940c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 0x7f6d940c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 -- 172.21.15.22:0/3156034682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009b30 con 0x7f6d940cc9e0 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d92ffd700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 0x7f6d940c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d927fc700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d9406c750 tx=0x7f6d88004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880048b0 con 0x7f6d940cc9e0 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d880092e0 con 0x7f6d940cc9e0 2023-06-23T18:04:07.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801b670 con 0x7f6d940cc9e0 2023-06-23T18:04:07.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d940c6ab0 con 0x7f6d940cc9e0 2023-06-23T18:04:07.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940c6760 con 0x7f6d940cc9e0 2023-06-23T18:04:07.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.103+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d940c6920 con 0x7f6d940cc9e0 2023-06-23T18:04:07.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.107+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6d8801b7d0 con 0x7f6d940cc9e0 2023-06-23T18:04:07.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.107+0000 7f6d7bfff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 0x7f6d800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.107+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6d88085960 con 0x7f6d940cc9e0 2023-06-23T18:04:07.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.107+0000 7f6d92ffd700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 0x7f6d800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.107+0000 7f6d92ffd700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 0x7f6d800687a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.115+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6d88052a60 con 0x7f6d940cc9e0 2023-06-23T18:04:07.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.315+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d9404e4c0 con 0x7f6d940cc9e0 2023-06-23T18:04:07.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.315+0000 7f6d7bfff700 1 -- 172.21.15.22:0/3156034682 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f6d88056080 con 0x7f6d940cc9e0 2023-06-23T18:04:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:07.331 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:07.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 msgr2=0x7f6d800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 0x7f6d800687a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).stop 2023-06-23T18:04:07.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 msgr2=0x7f6d9415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d9406c750 tx=0x7f6d88004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 shutdown_connections 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d940cf480 0x7f6d941557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d940cc9e0 0x7f6d9415d3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6d800662f0 0x7f6d800687a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 --2- 172.21.15.22:0/3156034682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d940b7af0 0x7f6d940c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 >> 172.21.15.22:0/3156034682 conn(0x7f6d94069960 msgr2=0x7f6d940b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 shutdown_connections 2023-06-23T18:04:07.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.319+0000 7f6d93fff700 1 -- 172.21.15.22:0/3156034682 wait complete. 2023-06-23T18:04:07.335 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:07.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 Processor -- start 2023-06-23T18:04:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 -- start start 2023-06-23T18:04:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea80070280 0x7fea800afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800c9570 0x7fea8006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800c9b50 con 0x7fea800affd0 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea801590a0 con 0x7fea80070280 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea86424700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800b0450 con 0x7fea800c9570 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea85422700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea80070280 0x7fea800afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea85422700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea80070280 0x7fea800afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50172/0 (socket says 172.21.15.22:50172) 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea85422700 1 -- 172.21.15.22:0/3612967584 learned_addr learned my addr 172.21.15.22:0/3612967584 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.443+0000 7fea85c23700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800c9570 0x7fea8006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea77fff700 1 -- 172.21.15.22:0/3612967584 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804824820 0 0) 0x7fea800c9b50 con 0x7fea800affd0 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea85422700 1 -- 172.21.15.22:0/3612967584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800c9570 msgr2=0x7fea8006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea85422700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800c9570 0x7fea8006b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea85422700 1 -- 172.21.15.22:0/3612967584 >> v1:172.21.15.169:6789/0 conn(0x7fea800affd0 legacy=0x7fea800c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea85422700 1 -- 172.21.15.22:0/3612967584 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800c9e30 con 0x7fea80070280 2023-06-23T18:04:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea85422700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea80070280 0x7fea800afa90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bbb0 tx=0x7fea7c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b82122f62f367e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea77fff700 1 -- 172.21.15.22:0/3612967584 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00dd30 con 0x7fea80070280 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea77fff700 1 -- 172.21.15.22:0/3612967584 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c00de90 con 0x7fea80070280 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea77fff700 1 -- 172.21.15.22:0/3612967584 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c010690 con 0x7fea80070280 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- 172.21.15.22:0/3612967584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea80070280 msgr2=0x7fea800afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea80070280 0x7fea800afa90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bbb0 tx=0x7fea7c00bec0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- 172.21.15.22:0/3612967584 shutdown_connections 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800c9570 0x7fea8006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 --2- 172.21.15.22:0/3612967584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea80070280 0x7fea800afa90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- 172.21.15.22:0/3612967584 >> 172.21.15.22:0/3612967584 conn(0x7fea8006a5f0 msgr2=0x7fea800caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- 172.21.15.22:0/3612967584 shutdown_connections 2023-06-23T18:04:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- 172.21.15.22:0/3612967584 wait complete. 2023-06-23T18:04:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 Processor -- start 2023-06-23T18:04:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.447+0000 7fea86424700 1 -- start start 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea80070280 0x7fea8014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800affd0 0x7fea800c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8014d570 con 0x7fea800c9570 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800adf00 con 0x7fea80070280 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800ae1e0 con 0x7fea800affd0 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50178/0 (socket says 172.21.15.22:50178) 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 -- 172.21.15.22:0/2312565509 learned_addr learned my addr 172.21.15.22:0/2312565509 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea84c21700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800affd0 0x7fea800c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85422700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea80070280 0x7fea8014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 -- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800affd0 msgr2=0x7fea800c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800affd0 0x7fea800c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 -- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea80070280 msgr2=0x7fea8014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea80070280 0x7fea8014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 -- 172.21.15.22:0/2312565509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c00b810 con 0x7fea800c9570 2023-06-23T18:04:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea85c23700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea7800efc0 tx=0x7fea7800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800cad0 con 0x7fea800c9570 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800f040 con 0x7fea800c9570 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea800ae4c0 con 0x7fea800c9570 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800ce70 con 0x7fea800c9570 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.451+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800aea80 con 0x7fea800c9570 2023-06-23T18:04:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800c2e80 con 0x7fea800c9570 2023-06-23T18:04:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fea78014480 con 0x7fea800c9570 2023-06-23T18:04:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea75ffb700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 0x7fea640687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea85422700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 0x7fea640687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fea78084ac0 con 0x7fea800c9570 2023-06-23T18:04:07.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.455+0000 7fea85422700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 0x7fea640687a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bfd0 tx=0x7fea7c013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.463+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fea800c2e80 con 0x7fea800c9570 2023-06-23T18:04:07.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.631+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea800c2e80 con 0x7fea800c9570 2023-06-23T18:04:07.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.631+0000 7fea75ffb700 1 -- 172.21.15.22:0/2312565509 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fea800c2e80 con 0x7fea800c9570 2023-06-23T18:04:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:07.650 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:07.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 msgr2=0x7fea640687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 0x7fea640687a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bfd0 tx=0x7fea7c013000 comp rx=0 tx=0).stop 2023-06-23T18:04:07.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 msgr2=0x7fea800b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea7800efc0 tx=0x7fea7800c0c0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 shutdown_connections 2023-06-23T18:04:07.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea800c9570 0x7fea800b31b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea800affd0 0x7fea800c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea640662f0 0x7fea640687a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 --2- 172.21.15.22:0/2312565509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea80070280 0x7fea8014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 >> 172.21.15.22:0/2312565509 conn(0x7fea8006a5f0 msgr2=0x7fea800abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 shutdown_connections 2023-06-23T18:04:07.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.635+0000 7fea86424700 1 -- 172.21.15.22:0/2312565509 wait complete. 2023-06-23T18:04:07.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:07.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.771+0000 7f0c15bc5700 1 Processor -- start 2023-06-23T18:04:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.771+0000 7f0c15bc5700 1 -- start start 2023-06-23T18:04:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c10069960 0x7f0c10069d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c100707b0 0x7f0c1015f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1006a270 con 0x7f0c10070c10 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c10071110 con 0x7f0c10069960 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1015c310 con 0x7f0c100707b0 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c10069960 0x7f0c10069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c153c4700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0c10070c10 0x7f0c1015fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57330/0 (socket says 172.21.15.22:57330) 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c153c4700 1 -- 172.21.15.22:0/4238113388 learned_addr learned my addr 172.21.15.22:0/4238113388 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c0ffff700 1 --2- 172.21.15.22:0/4238113388 >> v2:172.21.15.169:3300/0 conn(0x7f0c100707b0 0x7f0c1015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 -- 172.21.15.22:0/4238113388 >> v2:172.21.15.169:3300/0 conn(0x7f0c100707b0 msgr2=0x7f0c1015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 --2- 172.21.15.22:0/4238113388 >> v2:172.21.15.169:3300/0 conn(0x7f0c100707b0 0x7f0c1015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 -- 172.21.15.22:0/4238113388 >> v1:172.21.15.22:6789/0 conn(0x7f0c10070c10 legacy=0x7f0c1015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 -- 172.21.15.22:0/4238113388 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100ca720 con 0x7f0c10069960 2023-06-23T18:04:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c0f7fe700 1 -- 172.21.15.22:0/4238113388 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554455717 0 0) 0x7f0c1006a270 con 0x7f0c10070c10 2023-06-23T18:04:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c14bc3700 1 --2- 172.21.15.22:0/4238113388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10069960 0x7f0c10069d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c10070430 tx=0x7f0c04009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa3105804821b30 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c0f7fe700 1 -- 172.21.15.22:0/4238113388 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04011830 con 0x7f0c10069960 2023-06-23T18:04:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c0f7fe700 1 -- 172.21.15.22:0/4238113388 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04011990 con 0x7f0c10069960 2023-06-23T18:04:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c0f7fe700 1 -- 172.21.15.22:0/4238113388 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04011d20 con 0x7f0c10069960 2023-06-23T18:04:07.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- 172.21.15.22:0/4238113388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10069960 msgr2=0x7f0c10069d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/4238113388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10069960 0x7f0c10069d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c10070430 tx=0x7f0c04009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- 172.21.15.22:0/4238113388 shutdown_connections 2023-06-23T18:04:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/4238113388 >> v2:172.21.15.169:3300/0 conn(0x7f0c100707b0 0x7f0c1015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/4238113388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10069960 0x7f0c10069d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- 172.21.15.22:0/4238113388 >> 172.21.15.22:0/4238113388 conn(0x7f0c1006b700 msgr2=0x7f0c100b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- 172.21.15.22:0/4238113388 shutdown_connections 2023-06-23T18:04:07.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.775+0000 7f0c15bc5700 1 -- 172.21.15.22:0/4238113388 wait complete. 2023-06-23T18:04:07.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 Processor -- start 2023-06-23T18:04:07.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 -- start start 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c10069960 0x7f0c101f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c100b0e90 0x7f0c100c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ae320 con 0x7f0c10070c10 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100b29f0 con 0x7f0c100b0e90 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c15bc5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101f15e0 con 0x7f0c10069960 2023-06-23T18:04:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50204/0 (socket says 172.21.15.22:50204) 2023-06-23T18:04:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 -- 172.21.15.22:0/3660909948 learned_addr learned my addr 172.21.15.22:0/3660909948 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c14bc3700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c10069960 0x7f0c101f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c153c4700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c100b0e90 0x7f0c100c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 -- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c10069960 msgr2=0x7f0c101f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c10069960 0x7f0c101f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 -- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c100b0e90 msgr2=0x7f0c100c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c100b0e90 0x7f0c100c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 -- 172.21.15.22:0/3660909948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040094b0 con 0x7f0c10070c10 2023-06-23T18:04:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0ffff700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00ea00 tx=0x7f0bfc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.779+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc009720 con 0x7f0c10070c10 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00eea0 con 0x7f0c10070c10 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c100ad710 con 0x7f0c10070c10 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00ca50 con 0x7f0c10070c10 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c100adcd0 con 0x7f0c10070c10 2023-06-23T18:04:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1004e4c0 con 0x7f0c10070c10 2023-06-23T18:04:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0bfc00cbb0 con 0x7f0c10070c10 2023-06-23T18:04:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c0d7fa700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 0x7f0bf40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0bfc084350 con 0x7f0c10070c10 2023-06-23T18:04:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.783+0000 7f0c14bc3700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 0x7f0bf40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.787+0000 7f0c14bc3700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 0x7f0bf40687a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c040090b0 tx=0x7f0c04008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:07.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.791+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0bfc054ed0 con 0x7f0c10070c10 2023-06-23T18:04:07.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.955+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c100cba70 con 0x7f0c10070c10 2023-06-23T18:04:07.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.955+0000 7f0c0d7fa700 1 -- 172.21.15.22:0/3660909948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bfc054a60 con 0x7f0c10070c10 2023-06-23T18:04:07.971 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:07.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 msgr2=0x7f0bf40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 0x7f0bf40687a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c040090b0 tx=0x7f0c04008030 comp rx=0 tx=0).stop 2023-06-23T18:04:07.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 msgr2=0x7f0c100b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00ea00 tx=0x7f0bfc00ed10 comp rx=0 tx=0).stop 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 shutdown_connections 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0bf40662f0 0x7f0bf40687a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c100b0e90 0x7f0c100c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10070c10 0x7f0c100b0950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 --2- 172.21.15.22:0/3660909948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c10069960 0x7f0c101f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:07.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 >> 172.21.15.22:0/3660909948 conn(0x7f0c1006b700 msgr2=0x7f0c1015a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:07.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 shutdown_connections 2023-06-23T18:04:07.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:07.959+0000 7f0c15bc5700 1 -- 172.21.15.22:0/3660909948 wait complete. 2023-06-23T18:04:07.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.087+0000 7f5b63fff700 1 Processor -- start 2023-06-23T18:04:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.087+0000 7f5b63fff700 1 -- start start 2023-06-23T18:04:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b6406f660 0x7f5b640c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6415c2e0 0x7f5b640c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6406c6e0 con 0x7f5b6415bf10 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6406cf70 con 0x7f5b6406f660 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640cddf0 con 0x7f5b6415c2e0 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b6406f660 0x7f5b640c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b62ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5b6415bf10 0x7f5b6406bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57346/0 (socket says 172.21.15.22:57346) 2023-06-23T18:04:08.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b62ffd700 1 -- 172.21.15.22:0/537789386 learned_addr learned my addr 172.21.15.22:0/537789386 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b637fe700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6415c2e0 0x7f5b640c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 -- 172.21.15.22:0/537789386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6415c2e0 msgr2=0x7f5b640c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6415c2e0 0x7f5b640c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 -- 172.21.15.22:0/537789386 >> v1:172.21.15.22:6789/0 conn(0x7f5b6415bf10 legacy=0x7f5b6406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 -- 172.21.15.22:0/537789386 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6406c9b0 con 0x7f5b6406f660 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b627fc700 1 -- 172.21.15.22:0/537789386 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766402980 0 0) 0x7f5b6406c6e0 con 0x7f5b6415bf10 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b5bfff700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6406f660 0x7f5b640c89f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009850 tx=0x7f5b4c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecf2561a803bea6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b627fc700 1 -- 172.21.15.22:0/537789386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011830 con 0x7f5b6406f660 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b627fc700 1 -- 172.21.15.22:0/537789386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c011990 con 0x7f5b6406f660 2023-06-23T18:04:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b627fc700 1 -- 172.21.15.22:0/537789386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011d20 con 0x7f5b6406f660 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- 172.21.15.22:0/537789386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6406f660 msgr2=0x7f5b640c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6406f660 0x7f5b640c89f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009850 tx=0x7f5b4c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- 172.21.15.22:0/537789386 shutdown_connections 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6415c2e0 0x7f5b640c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 --2- 172.21.15.22:0/537789386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6406f660 0x7f5b640c89f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- 172.21.15.22:0/537789386 >> 172.21.15.22:0/537789386 conn(0x7f5b6406a5f0 msgr2=0x7f5b640caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- 172.21.15.22:0/537789386 shutdown_connections 2023-06-23T18:04:08.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.091+0000 7f5b63fff700 1 -- 172.21.15.22:0/537789386 wait complete. 2023-06-23T18:04:08.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 Processor -- start 2023-06-23T18:04:08.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 -- start start 2023-06-23T18:04:08.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6406f660 0x7f5b640c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 0x7f5b640ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b64099ad0 con 0x7f5b6415bf10 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6415a610 con 0x7f5b6415c2e0 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b63fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b64098e50 con 0x7f5b6406f660 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50228/0 (socket says 172.21.15.22:50228) 2023-06-23T18:04:08.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 -- 172.21.15.22:0/3132598618 learned_addr learned my addr 172.21.15.22:0/3132598618 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:08.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b62ffd700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6406f660 0x7f5b640c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b637fe700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 0x7f5b640ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 -- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6406f660 msgr2=0x7f5b640c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6406f660 0x7f5b640c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 -- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 msgr2=0x7f5b640ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 0x7f5b640ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 -- 172.21.15.22:0/3132598618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0094b0 con 0x7f5b6415bf10 2023-06-23T18:04:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b637fe700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 0x7f5b640ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5bfff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c014040 tx=0x7f5b4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.095+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0038c0 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c01e070 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b64099130 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c003c60 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b640996f0 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6404e4c0 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5b4c02d450 con 0x7f5b6415bf10 2023-06-23T18:04:08.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b5b7fe700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 0x7f5b480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b4c084f90 con 0x7f5b6415bf10 2023-06-23T18:04:08.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.099+0000 7f5b62ffd700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 0x7f5b480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.103+0000 7f5b62ffd700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 0x7f5b480687a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b54000f60 tx=0x7f5b54007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.107+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5b4c0521b0 con 0x7f5b6415bf10 2023-06-23T18:04:08.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.271+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b6406ae50 con 0x7f5b6415bf10 2023-06-23T18:04:08.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.271+0000 7f5b5b7fe700 1 -- 172.21.15.22:0/3132598618 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b4c0150a0 con 0x7f5b6415bf10 2023-06-23T18:04:08.288 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 msgr2=0x7f5b480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 0x7f5b480687a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b54000f60 tx=0x7f5b54007c00 comp rx=0 tx=0).stop 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 msgr2=0x7f5b6415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c014040 tx=0x7f5b4c004770 comp rx=0 tx=0).stop 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 shutdown_connections 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5b480662f0 0x7f5b480687a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b6415c2e0 0x7f5b640ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b6415bf10 0x7f5b6415a0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 --2- 172.21.15.22:0/3132598618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b6406f660 0x7f5b640c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 >> 172.21.15.22:0/3132598618 conn(0x7f5b6406a5f0 msgr2=0x7f5b640affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 shutdown_connections 2023-06-23T18:04:08.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.275+0000 7f5b63fff700 1 -- 172.21.15.22:0/3132598618 wait complete. 2023-06-23T18:04:08.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:08.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.407+0000 7f2d7a126700 1 Processor -- start 2023-06-23T18:04:08.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.407+0000 7f2d7a126700 1 -- start start 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d740d1f30 0x7f2d740d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d740d2360 0x7f2d740d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740ccb30 con 0x7f2d740cf3a0 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740d2880 con 0x7f2d740d1f30 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740d2b60 con 0x7f2d740d2360 2023-06-23T18:04:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d740d1f30 0x7f2d740d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d740d1f30 0x7f2d740d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50238/0 (socket says 172.21.15.22:50238) 2023-06-23T18:04:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d79925700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d740d2360 0x7f2d740d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 -- 172.21.15.22:0/2585286112 learned_addr learned my addr 172.21.15.22:0/2585286112 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 -- 172.21.15.22:0/2585286112 >> v2:172.21.15.169:3300/0 conn(0x7f2d740d2360 msgr2=0x7f2d740d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 --2- 172.21.15.22:0/2585286112 >> v2:172.21.15.169:3300/0 conn(0x7f2d740d2360 0x7f2d740d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 -- 172.21.15.22:0/2585286112 >> v1:172.21.15.169:6789/0 conn(0x7f2d740cf3a0 legacy=0x7f2d740cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 -- 172.21.15.22:0/2585286112 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740d2e40 con 0x7f2d740d1f30 2023-06-23T18:04:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d78923700 1 --2- 172.21.15.22:0/2585286112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740d1f30 0x7f2d740d0020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d7408bde0 tx=0x7f2d64009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ef2339c42a43a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d6bfff700 1 -- 172.21.15.22:0/2585286112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011830 con 0x7f2d740d1f30 2023-06-23T18:04:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d6bfff700 1 -- 172.21.15.22:0/2585286112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64011990 con 0x7f2d740d1f30 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d6bfff700 1 -- 172.21.15.22:0/2585286112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011d20 con 0x7f2d740d1f30 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- 172.21.15.22:0/2585286112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740d1f30 msgr2=0x7f2d740d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 --2- 172.21.15.22:0/2585286112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740d1f30 0x7f2d740d0020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d7408bde0 tx=0x7f2d64009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- 172.21.15.22:0/2585286112 shutdown_connections 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 --2- 172.21.15.22:0/2585286112 >> v2:172.21.15.169:3300/0 conn(0x7f2d740d2360 0x7f2d740d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 --2- 172.21.15.22:0/2585286112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740d1f30 0x7f2d740d0020 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- 172.21.15.22:0/2585286112 >> 172.21.15.22:0/2585286112 conn(0x7f2d7406a280 msgr2=0x7f2d7409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- 172.21.15.22:0/2585286112 shutdown_connections 2023-06-23T18:04:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.411+0000 7f2d7a126700 1 -- 172.21.15.22:0/2585286112 wait complete. 2023-06-23T18:04:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.415+0000 7f2d7a126700 1 Processor -- start 2023-06-23T18:04:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.415+0000 7f2d7a126700 1 -- start start 2023-06-23T18:04:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 0x7f2d741f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 0x7f2d740b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741f4f90 con 0x7f2d740cf3a0 2023-06-23T18:04:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741f51f0 con 0x7f2d740d2360 2023-06-23T18:04:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741f54d0 con 0x7f2d740d1f30 2023-06-23T18:04:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50248/0 (socket says 172.21.15.22:50248) 2023-06-23T18:04:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 -- 172.21.15.22:0/242438264 learned_addr learned my addr 172.21.15.22:0/242438264 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d78923700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 0x7f2d741f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79925700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 0x7f2d740b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 -- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 msgr2=0x7f2d741f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 0x7f2d741f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 -- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 msgr2=0x7f2d740b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 0x7f2d740b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 -- 172.21.15.22:0/242438264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009570 con 0x7f2d740cf3a0 2023-06-23T18:04:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79925700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 0x7f2d740b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d78923700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 0x7f2d741f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d79124700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001f00 tx=0x7f2d7000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700052d0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70005430 con 0x7f2d740cf3a0 2023-06-23T18:04:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d741f57b0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70006af0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d740b7f50 con 0x7f2d740cf3a0 2023-06-23T18:04:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.419+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d740b80e0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.423+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2d700036f0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.423+0000 7f2d69ffb700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 0x7f2d580687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.423+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2d700837e0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.423+0000 7f2d78923700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 0x7f2d580687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.423+0000 7f2d78923700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 0x7f2d580687d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d64008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.427+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2d700509e0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d7404e4c0 con 0x7f2d740cf3a0 2023-06-23T18:04:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d69ffb700 1 -- 172.21.15.22:0/242438264 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d70054000 con 0x7f2d740cf3a0 2023-06-23T18:04:08.606 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:08.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 msgr2=0x7f2d580687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 0x7f2d580687d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d64008030 comp rx=0 tx=0).stop 2023-06-23T18:04:08.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 msgr2=0x7f2d740b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001f00 tx=0x7f2d7000c860 comp rx=0 tx=0).stop 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 shutdown_connections 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d58066320 0x7f2d580687d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d740d2360 0x7f2d740b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d740d1f30 0x7f2d741f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 --2- 172.21.15.22:0/242438264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d740cf3a0 0x7f2d740b6fc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 >> 172.21.15.22:0/242438264 conn(0x7f2d7406a280 msgr2=0x7f2d740b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 shutdown_connections 2023-06-23T18:04:08.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.591+0000 7f2d7a126700 1 -- 172.21.15.22:0/242438264 wait complete. 2023-06-23T18:04:08.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 Processor -- start 2023-06-23T18:04:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- start start 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff89c0afaf0 0x7ff89c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c0c8a30 con 0x7ff89c158370 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c0c8d10 con 0x7ff89c06f860 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c158740 con 0x7ff89c0afaf0 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff89c0afaf0 0x7ff89c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a0c3d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff89c158370 0x7ff89c15c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57350/0 (socket says 172.21.15.22:57350) 2023-06-23T18:04:08.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a0c3d700 1 -- 172.21.15.22:0/1390045560 learned_addr learned my addr 172.21.15.22:0/1390045560 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:08.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89b7fe700 1 -- 172.21.15.22:0/1390045560 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063385437 0 0) 0x7ff89c0c8a30 con 0x7ff89c158370 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89b7fe700 1 -- 172.21.15.22:0/1390045560 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8880034f0 con 0x7ff89c158370 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89bfff700 1 -- 172.21.15.22:0/1390045560 >> v1:172.21.15.169:6789/0 conn(0x7ff89c06f860 legacy=0x7ff89c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89bfff700 1 -- 172.21.15.22:0/1390045560 >> v1:172.21.15.22:6789/0 conn(0x7ff89c158370 legacy=0x7ff89c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89bfff700 1 -- 172.21.15.22:0/1390045560 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c0ad070 con 0x7ff89c0afaf0 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89bfff700 1 --2- 172.21.15.22:0/1390045560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0aff70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5588aad1a9153cd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89b7fe700 1 -- 172.21.15.22:0/1390045560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00e070 con 0x7ff89c0afaf0 2023-06-23T18:04:08.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89b7fe700 1 -- 172.21.15.22:0/1390045560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00bae0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff89b7fe700 1 -- 172.21.15.22:0/1390045560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00be50 con 0x7ff89c0afaf0 2023-06-23T18:04:08.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1390045560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 msgr2=0x7ff89c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1390045560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0aff70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:08.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1390045560 shutdown_connections 2023-06-23T18:04:08.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1390045560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0aff70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1390045560 >> 172.21.15.22:0/1390045560 conn(0x7ff89c06a5f0 msgr2=0x7ff89c0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1390045560 shutdown_connections 2023-06-23T18:04:08.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.731+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1390045560 wait complete. 2023-06-23T18:04:08.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 Processor -- start 2023-06-23T18:04:08.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 -- start start 2023-06-23T18:04:08.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff89c06f860 0x7ff89c09ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff89c158370 0x7ff89c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c09b310 con 0x7ff89c0afaf0 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c09b5f0 con 0x7ff89c06f860 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a1c3f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c09b8d0 con 0x7ff89c158370 2023-06-23T18:04:08.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50274/0 (socket says 172.21.15.22:50274) 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 -- 172.21.15.22:0/1494527202 learned_addr learned my addr 172.21.15.22:0/1494527202 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a0c3d700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff89c06f860 0x7ff89c09ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff8a143e700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff89c158370 0x7ff89c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 -- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff89c158370 msgr2=0x7ff89c06fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff89c158370 0x7ff89c06fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 -- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff89c06f860 msgr2=0x7ff89c09ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff89c06f860 0x7ff89c09ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 -- 172.21.15.22:0/1494527202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c009a70 con 0x7ff89c0afaf0 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.735+0000 7ff89bfff700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00d010 tx=0x7ff88c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0048b0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c009410 con 0x7ff89c0afaf0 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0266a0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0c6770 con 0x7ff89c0afaf0 2023-06-23T18:04:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c0c6420 con 0x7ff89c0afaf0 2023-06-23T18:04:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c04e4c0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff88c012430 con 0x7ff89c0afaf0 2023-06-23T18:04:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 0x7ff87c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff88c085e80 con 0x7ff89c0afaf0 2023-06-23T18:04:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.739+0000 7ff8a0c3d700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 0x7ff87c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.743+0000 7ff8a0c3d700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 0x7ff87c0687a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff884007c60 tx=0x7ff884007bb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.747+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff88c0530a0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.911+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff89c0b3ff0 con 0x7ff89c0afaf0 2023-06-23T18:04:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.911+0000 7ff8997fa700 1 -- 172.21.15.22:0/1494527202 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff88c021020 con 0x7ff89c0afaf0 2023-06-23T18:04:08.926 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 msgr2=0x7ff87c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 0x7ff87c0687a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff884007c60 tx=0x7ff884007bb0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 msgr2=0x7ff89c0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00d010 tx=0x7ff88c004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 shutdown_connections 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff89c158370 0x7ff89c06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff89c0afaf0 0x7ff89c0b3a90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff87c0662f0 0x7ff87c0687a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 --2- 172.21.15.22:0/1494527202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff89c06f860 0x7ff89c09ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 >> 172.21.15.22:0/1494527202 conn(0x7ff89c06a5f0 msgr2=0x7ff89c0adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 shutdown_connections 2023-06-23T18:04:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:08.915+0000 7ff8a1c3f700 1 -- 172.21.15.22:0/1494527202 wait complete. 2023-06-23T18:04:08.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.047+0000 7f1e0e5f4700 1 Processor -- start 2023-06-23T18:04:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.047+0000 7f1e0e5f4700 1 -- start start 2023-06-23T18:04:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080b7dc0 0x7f1e0815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e080b1a00 0x7f1e080ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e080d0bd0 con 0x7f1e080cd120 2023-06-23T18:04:09.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e080b8190 con 0x7f1e080b1a00 2023-06-23T18:04:09.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080ccc40 con 0x7f1e080b7dc0 2023-06-23T18:04:09.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e080b1a00 0x7f1e080ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1dff7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1e080cd120 0x7f1e0815fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57366/0 (socket says 172.21.15.22:57366) 2023-06-23T18:04:09.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1dff7fe700 1 -- 172.21.15.22:0/4288963955 learned_addr learned my addr 172.21.15.22:0/4288963955 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:09.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e07fff700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080b7dc0 0x7f1e0815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e077fe700 1 -- 172.21.15.22:0/4288963955 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153763393 0 0) 0x7f1e080d0bd0 con 0x7f1e080cd120 2023-06-23T18:04:09.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e077fe700 1 -- 172.21.15.22:0/4288963955 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de80035d0 con 0x7f1e080cd120 2023-06-23T18:04:09.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 -- 172.21.15.22:0/4288963955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080b7dc0 msgr2=0x7f1e0815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:09.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080b7dc0 0x7f1e0815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 -- 172.21.15.22:0/4288963955 >> v1:172.21.15.22:6789/0 conn(0x7f1e080cd120 legacy=0x7f1e0815fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 -- 172.21.15.22:0/4288963955 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080d2140 con 0x7f1e080b1a00 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0cb91700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e080ce840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009d90 tx=0x7f1e0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8528b8e4549b113 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e077fe700 1 -- 172.21.15.22:0/4288963955 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0000e070 con 0x7f1e080b1a00 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e077fe700 1 -- 172.21.15.22:0/4288963955 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0000bb20 con 0x7f1e080b1a00 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e077fe700 1 -- 172.21.15.22:0/4288963955 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00012430 con 0x7f1e080b1a00 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/4288963955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 msgr2=0x7f1e080ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e080ce840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009d90 tx=0x7f1e0000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:09.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/4288963955 shutdown_connections 2023-06-23T18:04:09.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e080ce840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/4288963955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080b7dc0 0x7f1e0815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/4288963955 >> 172.21.15.22:0/4288963955 conn(0x7f1e0806b3e0 msgr2=0x7f1e0806b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/4288963955 shutdown_connections 2023-06-23T18:04:09.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.051+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/4288963955 wait complete. 2023-06-23T18:04:09.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 Processor -- start 2023-06-23T18:04:09.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- start start 2023-06-23T18:04:09.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e080b7dc0 0x7f1e08153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 0x7f1e080c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0815ee50 con 0x7f1e080b1a00 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0815dc70 con 0x7f1e080b7dc0 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080c9b60 con 0x7f1e080cd120 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50290/0 (socket says 172.21.15.22:50290) 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 -- 172.21.15.22:0/3295640764 learned_addr learned my addr 172.21.15.22:0/3295640764 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1dff7fe700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e080b7dc0 0x7f1e08153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0cb91700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 0x7f1e080c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 -- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 msgr2=0x7f1e080c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 0x7f1e080c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 -- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e080b7dc0 msgr2=0x7f1e08153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e080b7dc0 0x7f1e08153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 -- 172.21.15.22:0/3295640764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00009a70 con 0x7f1e080b1a00 2023-06-23T18:04:09.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0cb91700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 0x7f1e080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e07fff700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e840 tx=0x7f1df800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80095a0 con 0x7f1e080b1a00 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8013680 con 0x7f1e080b1a00 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0815e480 con 0x7f1e080b1a00 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80139f0 con 0x7f1e080b1a00 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.055+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0815df70 con 0x7f1e080b1a00 2023-06-23T18:04:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0804dc70 con 0x7f1e080b1a00 2023-06-23T18:04:09.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1df8013b50 con 0x7f1e080b1a00 2023-06-23T18:04:09.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1e057fa700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 0x7f1dec0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1df8083e70 con 0x7f1e080b1a00 2023-06-23T18:04:09.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1dff7fe700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 0x7f1dec0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.059+0000 7f1dff7fe700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 0x7f1dec0687d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e081f4a00 tx=0x7f1df0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.067+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1df8054b20 con 0x7f1e080b1a00 2023-06-23T18:04:09.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.227+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e080b4ff0 con 0x7f1e080b1a00 2023-06-23T18:04:09.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.227+0000 7f1e057fa700 1 -- 172.21.15.22:0/3295640764 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1df8017020 con 0x7f1e080b1a00 2023-06-23T18:04:09.243 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:09.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 msgr2=0x7f1dec0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 0x7f1dec0687d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e081f4a00 tx=0x7f1df0007de0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 msgr2=0x7f1e081535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e840 tx=0x7f1df800eb50 comp rx=0 tx=0).stop 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 shutdown_connections 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e080cd120 0x7f1e080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1dec066320 0x7f1dec0687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e080b7dc0 0x7f1e08153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 --2- 172.21.15.22:0/3295640764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e080b1a00 0x7f1e081535d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 >> 172.21.15.22:0/3295640764 conn(0x7f1e0806b3e0 msgr2=0x7f1e0815aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 shutdown_connections 2023-06-23T18:04:09.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.231+0000 7f1e0e5f4700 1 -- 172.21.15.22:0/3295640764 wait complete. 2023-06-23T18:04:09.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.355+0000 7fc6be5db700 1 Processor -- start 2023-06-23T18:04:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- start start 2023-06-23T18:04:09.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81580e0 0x7fc6b80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc6b80cd310 0x7fc6b80713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b8071930 con 0x7fc6b81584b0 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80cc270 con 0x7fc6b80cd310 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80cc550 con 0x7fc6b81580e0 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc6b80cd310 0x7fc6b80713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc6b80cd310 0x7fc6b80713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50304/0 (socket says 172.21.15.22:50304) 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 -- 172.21.15.22:0/3424485517 learned_addr learned my addr 172.21.15.22:0/3424485517 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bd5d9700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81580e0 0x7fc6b80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 -- 172.21.15.22:0/3424485517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81580e0 msgr2=0x7fc6b80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81580e0 0x7fc6b80cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 -- 172.21.15.22:0/3424485517 >> v1:172.21.15.169:6789/0 conn(0x7fc6b81584b0 legacy=0x7fc6b8158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 -- 172.21.15.22:0/3424485517 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80cc830 con 0x7fc6b80cd310 2023-06-23T18:04:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6bcdd8700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b80713f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009910 tx=0x7fc6a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82361ee955787b75 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6affff700 1 -- 172.21.15.22:0/3424485517 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8011830 con 0x7fc6b80cd310 2023-06-23T18:04:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6affff700 1 -- 172.21.15.22:0/3424485517 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8011990 con 0x7fc6b80cd310 2023-06-23T18:04:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6affff700 1 -- 172.21.15.22:0/3424485517 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8011d40 con 0x7fc6b80cd310 2023-06-23T18:04:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- 172.21.15.22:0/3424485517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 msgr2=0x7fc6b80713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b80713f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009910 tx=0x7fc6a8009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- 172.21.15.22:0/3424485517 shutdown_connections 2023-06-23T18:04:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b80713f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 --2- 172.21.15.22:0/3424485517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81580e0 0x7fc6b80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- 172.21.15.22:0/3424485517 >> 172.21.15.22:0/3424485517 conn(0x7fc6b806a5f0 msgr2=0x7fc6b80caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- 172.21.15.22:0/3424485517 shutdown_connections 2023-06-23T18:04:09.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.359+0000 7fc6be5db700 1 -- 172.21.15.22:0/3424485517 wait complete. 2023-06-23T18:04:09.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 Processor -- start 2023-06-23T18:04:09.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 -- start start 2023-06-23T18:04:09.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b81580e0 0x7fc6b80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81584b0 0x7fc6b80aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80c3440 con 0x7fc6b80cd310 2023-06-23T18:04:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80c3720 con 0x7fc6b81580e0 2023-06-23T18:04:09.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6be5db700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80c3a00 con 0x7fc6b81584b0 2023-06-23T18:04:09.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50320/0 (socket says 172.21.15.22:50320) 2023-06-23T18:04:09.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 -- 172.21.15.22:0/291569625 learned_addr learned my addr 172.21.15.22:0/291569625 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:09.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bddda700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81584b0 0x7fc6b80aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bcdd8700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b81580e0 0x7fc6b80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 -- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81584b0 msgr2=0x7fc6b80aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81584b0 0x7fc6b80aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 -- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b81580e0 msgr2=0x7fc6b80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b81580e0 0x7fc6b80ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.363+0000 7fc6bd5d9700 1 -- 172.21.15.22:0/291569625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009570 con 0x7fc6b80cd310 2023-06-23T18:04:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6bd5d9700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e9d0 tx=0x7fc6b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4009720 con 0x7fc6b80cd310 2023-06-23T18:04:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400ee70 con 0x7fc6b80cd310 2023-06-23T18:04:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400ca50 con 0x7fc6b80cd310 2023-06-23T18:04:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b80c3ce0 con 0x7fc6b80cd310 2023-06-23T18:04:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b8151ec0 con 0x7fc6b80cd310 2023-06-23T18:04:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b804e4c0 con 0x7fc6b80cd310 2023-06-23T18:04:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc6b400cbb0 con 0x7fc6b80cd310 2023-06-23T18:04:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 0x7fc69c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.367+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc6b4083f70 con 0x7fc6b80cd310 2023-06-23T18:04:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.371+0000 7fc6bcdd8700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 0x7fc69c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.371+0000 7fc6bcdd8700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 0x7fc69c0687a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8000c00 tx=0x7fc6a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.375+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc6b40b4700 con 0x7fc6b80cd310 2023-06-23T18:04:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6b80b05b0 con 0x7fc6b80cd310 2023-06-23T18:04:09.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6adffb700 1 -- 172.21.15.22:0/291569625 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fc6b4017090 con 0x7fc6b80cd310 2023-06-23T18:04:09.590 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:09.590 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 msgr2=0x7fc69c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 0x7fc69c0687a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8000c00 tx=0x7fc6a8008030 comp rx=0 tx=0).stop 2023-06-23T18:04:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 msgr2=0x7fc6b8151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e9d0 tx=0x7fc6b400ece0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 shutdown_connections 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6b81584b0 0x7fc6b80aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc69c0662f0 0x7fc69c0687a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b81580e0 0x7fc6b80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 --2- 172.21.15.22:0/291569625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6b80cd310 0x7fc6b8151980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 >> 172.21.15.22:0/291569625 conn(0x7fc6b806a5f0 msgr2=0x7fc6b80abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 shutdown_connections 2023-06-23T18:04:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.575+0000 7fc6be5db700 1 -- 172.21.15.22:0/291569625 wait complete. 2023-06-23T18:04:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:09.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:09.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 Processor -- start 2023-06-23T18:04:09.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 -- start start 2023-06-23T18:04:09.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f228015ff90 0x7f22800c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2280164160 0x7f22800c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22800cc360 con 0x7f22800ccd70 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22800cdb60 con 0x7f2280164160 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227f59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2280160360 con 0x7f228015ff90 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2280164160 0x7f22800c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2280164160 0x7f22800c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50328/0 (socket says 172.21.15.22:50328) 2023-06-23T18:04:09.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 -- 172.21.15.22:0/1744673715 learned_addr learned my addr 172.21.15.22:0/1744673715 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:09.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227e59c700 1 --2- 172.21.15.22:0/1744673715 >> v2:172.21.15.169:3300/0 conn(0x7f228015ff90 0x7f22800c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 -- 172.21.15.22:0/1744673715 >> v2:172.21.15.169:3300/0 conn(0x7f228015ff90 msgr2=0x7f22800c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 --2- 172.21.15.22:0/1744673715 >> v2:172.21.15.169:3300/0 conn(0x7f228015ff90 0x7f22800c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 -- 172.21.15.22:0/1744673715 >> v1:172.21.15.169:6789/0 conn(0x7f22800ccd70 legacy=0x7f2280163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.707+0000 7f227ed9d700 1 -- 172.21.15.22:0/1744673715 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22800c3fe0 con 0x7f2280164160 2023-06-23T18:04:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227ed9d700 1 --2- 172.21.15.22:0/1744673715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2280164160 0x7f22800c9f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2280072110 tx=0x7f227800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1b918f1174efda server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227d59a700 1 -- 172.21.15.22:0/1744673715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800e070 con 0x7f2280164160 2023-06-23T18:04:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227d59a700 1 -- 172.21.15.22:0/1744673715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227800b910 con 0x7f2280164160 2023-06-23T18:04:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227d59a700 1 -- 172.21.15.22:0/1744673715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800bcc0 con 0x7f2280164160 2023-06-23T18:04:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- 172.21.15.22:0/1744673715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2280164160 msgr2=0x7f22800c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 --2- 172.21.15.22:0/1744673715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2280164160 0x7f22800c9f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2280072110 tx=0x7f227800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- 172.21.15.22:0/1744673715 shutdown_connections 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 --2- 172.21.15.22:0/1744673715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2280164160 0x7f22800c9f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 --2- 172.21.15.22:0/1744673715 >> v2:172.21.15.169:3300/0 conn(0x7f228015ff90 0x7f22800c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- 172.21.15.22:0/1744673715 >> 172.21.15.22:0/1744673715 conn(0x7f22800cb2a0 msgr2=0x7f22800cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- 172.21.15.22:0/1744673715 shutdown_connections 2023-06-23T18:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- 172.21.15.22:0/1744673715 wait complete. 2023-06-23T18:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 Processor -- start 2023-06-23T18:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.711+0000 7f227f59e700 1 -- start start 2023-06-23T18:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800ccd70 0x7f2280151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 0x7f228015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22801e9af0 con 0x7f228015ff90 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22801e9d40 con 0x7f2280164160 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22801ea020 con 0x7f22800ccd70 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50334/0 (socket says 172.21.15.22:50334) 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 -- 172.21.15.22:0/2702701209 learned_addr learned my addr 172.21.15.22:0/2702701209 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:09.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227e59c700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800ccd70 0x7f2280151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227ed9d700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 0x7f228015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 -- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800ccd70 msgr2=0x7f2280151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800ccd70 0x7f2280151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 -- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 msgr2=0x7f228015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 0x7f228015a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 -- 172.21.15.22:0/2702701209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278009b30 con 0x7f228015ff90 2023-06-23T18:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227ed9d700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 0x7f228015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227dd9b700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f227400c500 tx=0x7f227400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740052d0 con 0x7f228015ff90 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274005430 con 0x7f228015ff90 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22801ea300 con 0x7f228015ff90 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740055a0 con 0x7f228015ff90 2023-06-23T18:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22801ea8c0 con 0x7f228015ff90 2023-06-23T18:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.715+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228004e4c0 con 0x7f228015ff90 2023-06-23T18:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.719+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f22740036a0 con 0x7f228015ff90 2023-06-23T18:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.719+0000 7f226effd700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 0x7f22600687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.719+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2274083290 con 0x7f228015ff90 2023-06-23T18:04:09.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.719+0000 7f227e59c700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 0x7f22600687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:09.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.719+0000 7f227e59c700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 0x7f22600687d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f2270008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:09.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.723+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f22740504b0 con 0x7f228015ff90 2023-06-23T18:04:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.891+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f228015b5d0 con 0x7f228015ff90 2023-06-23T18:04:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.895+0000 7f226effd700 1 -- 172.21.15.22:0/2702701209 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f2274053ad0 con 0x7f228015ff90 2023-06-23T18:04:09.911 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:09.911 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.895+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 msgr2=0x7f22600687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.895+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 0x7f22600687d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f2270008040 comp rx=0 tx=0).stop 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.895+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 msgr2=0x7f2280159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f227400c500 tx=0x7f227400c8c0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 shutdown_connections 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2260066320 0x7f22600687d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280164160 0x7f228015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f228015ff90 0x7f2280159b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 --2- 172.21.15.22:0/2702701209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800ccd70 0x7f2280151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 >> 172.21.15.22:0/2702701209 conn(0x7f22800cb2a0 msgr2=0x7f22800c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 shutdown_connections 2023-06-23T18:04:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:09.899+0000 7f227f59e700 1 -- 172.21.15.22:0/2702701209 wait complete. 2023-06-23T18:04:09.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:10.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 Processor -- start 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 -- start start 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00fc0cfe10 0x7f00fc15f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00fc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0cd8c0 con 0x7f00fc0cc450 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc06cb30 con 0x7f00fc0cfe10 2023-06-23T18:04:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0102619700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15c100 con 0x7f00fc06c150 2023-06-23T18:04:10.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0100e16700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f00fc0cc450 0x7f00fc0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57382/0 (socket says 172.21.15.22:57382) 2023-06-23T18:04:10.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0100e16700 1 -- 172.21.15.22:0/1596425338 learned_addr learned my addr 172.21.15.22:0/1596425338 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:10.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0101617700 1 --2- 172.21.15.22:0/1596425338 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0cfe10 0x7f00fc15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f0101e18700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00fc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785859131 0 0) 0x7f00fc0cd8c0 con 0x7f00fc0cc450 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.027+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e40035d0 con 0x7f00fc0cc450 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581976696 0 0) 0x7f00e40035d0 con 0x7f00fc0cc450 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 msgr2=0x7f00fc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00fc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0cfe10 msgr2=0x7f00fc15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 --2- 172.21.15.22:0/1596425338 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0cfe10 0x7f00fc15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc160110 con 0x7f00fc0cc450 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f00f0001670 con 0x7f00fc0cc450 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 >> v1:172.21.15.22:6789/0 conn(0x7f00fc0cc450 legacy=0x7f00fc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e4003a20 0x7f00e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00e4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4004db0 0x7f00e4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15c100 con 0x7f00e4003a20 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400ba30 con 0x7f00e4004db0 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400bb70 con 0x7f00fc06c150 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e4003a20 0x7f00e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0100e16700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00e4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101e18700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4004db0 0x7f00e4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 -- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 msgr2=0x7f00e4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06c150 0x7f00e4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 -- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4004db0 msgr2=0x7f00e4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4004db0 0x7f00e4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 -- 172.21.15.22:0/1596425338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40035d0 con 0x7f00e4003a20 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101617700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e4003a20 0x7f00e4003e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009fc0 tx=0x7f00e40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d7f03c1e8e662 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400ebd0 con 0x7f00e4003a20 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x7f00e4003a20 2023-06-23T18:04:10.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f00fbfff700 1 -- 172.21.15.22:0/1596425338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4014480 con 0x7f00e4003a20 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 -- 172.21.15.22:0/1596425338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e4003a20 msgr2=0x7f00e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 --2- 172.21.15.22:0/1596425338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e4003a20 0x7f00e4003e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009fc0 tx=0x7f00e40058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0101e18700 1 -- 172.21.15.22:0/1596425338 reap_dead start 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 -- 172.21.15.22:0/1596425338 shutdown_connections 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 -- 172.21.15.22:0/1596425338 >> 172.21.15.22:0/1596425338 conn(0x7f00fc069960 msgr2=0x7f00fc069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 -- 172.21.15.22:0/1596425338 shutdown_connections 2023-06-23T18:04:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 -- 172.21.15.22:0/1596425338 wait complete. 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.031+0000 7f0102619700 1 Processor -- start 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- start start 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4003a20 0x7f00fc0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 0x7f00fc0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ab8f0 con 0x7f00fc0ad990 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0c8ed0 con 0x7f00e4003a20 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15d1c0 con 0x7f00fc15e810 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50362/0 (socket says 172.21.15.22:50362) 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 -- 172.21.15.22:0/3463867200 learned_addr learned my addr 172.21.15.22:0/3463867200 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101617700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4003a20 0x7f00fc0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0100e16700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 0x7f00fc0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 -- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 msgr2=0x7f00fc0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 0x7f00fc0ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 -- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4003a20 msgr2=0x7f00fc0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4003a20 0x7f00fc0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 -- 172.21.15.22:0/3463867200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0abb00 con 0x7f00fc0ad990 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0100e16700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 0x7f00fc0ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0101e18700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cc370 tx=0x7f00ec010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec004300 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec004460 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc0abde0 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0045d0 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.035+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0c98a0 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0c9060 con 0x7f00fc0ad990 2023-06-23T18:04:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f00ec003900 con 0x7f00fc0ad990 2023-06-23T18:04:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f00f9ffb700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 0x7f00e80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f00ec083c00 con 0x7f00fc0ad990 2023-06-23T18:04:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f0101617700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 0x7f00e80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.039+0000 7f0101617700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 0x7f00e80687d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000f80 tx=0x7f00f4013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.047+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f00ec08a080 con 0x7f00fc0ad990 2023-06-23T18:04:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.252+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00fc0abfd0 con 0x7f00fc0ad990 2023-06-23T18:04:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.252+0000 7f00f9ffb700 1 -- 172.21.15.22:0/3463867200 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f00ec050e20 con 0x7f00fc0ad990 2023-06-23T18:04:10.268 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:10.268 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 msgr2=0x7f00e80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 0x7f00e80687d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000f80 tx=0x7f00f4013000 comp rx=0 tx=0).stop 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 msgr2=0x7f00fc0c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cc370 tx=0x7f00ec010ad0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 shutdown_connections 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00fc0ad990 0x7f00fc0c9360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc15e810 0x7f00fc0ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f00e8066320 0x7f00e80687d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 --2- 172.21.15.22:0/3463867200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e4003a20 0x7f00fc0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 >> 172.21.15.22:0/3463867200 conn(0x7f00fc069960 msgr2=0x7f00fc0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 shutdown_connections 2023-06-23T18:04:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.256+0000 7f0102619700 1 -- 172.21.15.22:0/3463867200 wait complete. 2023-06-23T18:04:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:10.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-23T18:03:37.725874+0000', 'modified': '2023-06-23T18:04:00.692712+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6837/2486746630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6835', 'nonce': 2486746630}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 2486746630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:04:10.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-23T18:04:10.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-23T18:04:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.804+0000 7f0edf351700 1 Processor -- start 2023-06-23T18:04:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 -- start start 2023-06-23T18:04:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ed8152dc0 0x7f0ed8153190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80cb000 0x7f0ed815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80c9330 con 0x7f0ed80c97c0 2023-06-23T18:04:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80c9c10 con 0x7f0ed8152dc0 2023-06-23T18:04:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edf351700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80c8a20 con 0x7f0ed80cb000 2023-06-23T18:04:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ed8152dc0 0x7f0ed8153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ed8152dc0 0x7f0ed8153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50368/0 (socket says 172.21.15.22:50368) 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1346062214 learned_addr learned my addr 172.21.15.22:0/1346062214 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80cb000 0x7f0ed815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1346062214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80cb000 msgr2=0x7f0ed815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80cb000 0x7f0ed815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0ecffff700 1 -- 172.21.15.22:0/1346062214 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321717162 0 0) 0x7f0ed80c9330 con 0x7f0ed80c97c0 2023-06-23T18:04:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1346062214 >> v1:172.21.15.169:6789/0 conn(0x7f0ed80c97c0 legacy=0x7f0ed80ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1346062214 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed815d4d0 con 0x7f0ed8152dc0 2023-06-23T18:04:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed8152dc0 0x7f0ed8153190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009910 tx=0x7f0ec8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45021f6008e6f70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0ecffff700 1 -- 172.21.15.22:0/1346062214 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011830 con 0x7f0ed8152dc0 2023-06-23T18:04:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0ecffff700 1 -- 172.21.15.22:0/1346062214 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8011990 con 0x7f0ed8152dc0 2023-06-23T18:04:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.808+0000 7f0ecffff700 1 -- 172.21.15.22:0/1346062214 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011d40 con 0x7f0ed8152dc0 2023-06-23T18:04:10.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- 172.21.15.22:0/1346062214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed8152dc0 msgr2=0x7f0ed8153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed8152dc0 0x7f0ed8153190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009910 tx=0x7f0ec8009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:10.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- 172.21.15.22:0/1346062214 shutdown_connections 2023-06-23T18:04:10.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80cb000 0x7f0ed815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 --2- 172.21.15.22:0/1346062214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed8152dc0 0x7f0ed8153190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- 172.21.15.22:0/1346062214 >> 172.21.15.22:0/1346062214 conn(0x7f0ed8069960 msgr2=0x7f0ed8069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- 172.21.15.22:0/1346062214 shutdown_connections 2023-06-23T18:04:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- 172.21.15.22:0/1346062214 wait complete. 2023-06-23T18:04:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 Processor -- start 2023-06-23T18:04:10.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.812+0000 7f0edf351700 1 -- start start 2023-06-23T18:04:10.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80cb000 0x7f0ed80adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed8152dc0 0x7f0ed80ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80af930 con 0x7f0ed80c97c0 2023-06-23T18:04:10.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80ae890 con 0x7f0ed80cb000 2023-06-23T18:04:10.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80aeb70 con 0x7f0ed8152dc0 2023-06-23T18:04:10.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50382/0 (socket says 172.21.15.22:50382) 2023-06-23T18:04:10.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 -- 172.21.15.22:0/747273948 learned_addr learned my addr 172.21.15.22:0/747273948 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed8152dc0 0x7f0ed80ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80cb000 0x7f0ed80adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 -- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed8152dc0 msgr2=0x7f0ed80ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed8152dc0 0x7f0ed80ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 -- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80cb000 msgr2=0x7f0ed80adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80cb000 0x7f0ed80adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 -- 172.21.15.22:0/747273948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8009570 con 0x7f0ed80c97c0 2023-06-23T18:04:10.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80cb490 tx=0x7f0ec8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0ecdffb700 1 -- 172.21.15.22:0/747273948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004380 con 0x7f0ed80c97c0 2023-06-23T18:04:10.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0ecdffb700 1 -- 172.21.15.22:0/747273948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8021070 con 0x7f0ed80c97c0 2023-06-23T18:04:10.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed80aee50 con 0x7f0ed80c97c0 2023-06-23T18:04:10.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0ecdffb700 1 -- 172.21.15.22:0/747273948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80036c0 con 0x7f0ed80c97c0 2023-06-23T18:04:10.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.816+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed80af410 con 0x7f0ed80c97c0 2023-06-23T18:04:10.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.820+0000 7f0ecdffb700 1 -- 172.21.15.22:0/747273948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0ec8023030 con 0x7f0ed80c97c0 2023-06-23T18:04:10.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.820+0000 7f0ecdffb700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 0x7f0ebc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.820+0000 7f0ecdffb700 1 -- 172.21.15.22:0/747273948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ec80875f0 con 0x7f0ed80c97c0 2023-06-23T18:04:10.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.820+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 0x7f0ebc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.820+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 0x7f0ebc0687a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8071c80 tx=0x7f0ed4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 Processor -- start 2023-06-23T18:04:10.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 -- start start 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1ea08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 0x1e55c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d74690 con 0x1be7eb0 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dc77c0 con 0x1be7710 2023-06-23T18:04:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dc0a20 con 0x1be7ae0 2023-06-23T18:04:10.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50386/0 (socket says 172.21.15.22:50386) 2023-06-23T18:04:10.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2318541686 learned_addr learned my addr 172.21.15.22:0/2318541686 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1ea08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.824+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 0x1e55c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 msgr2=0x1ea08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1ea08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 msgr2=0x1e55c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 0x1e55c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2318541686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c339e0 con 0x1be7eb0 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 0x1e55c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8085670 tx=0x7f0ec8010380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51082c939b2db6da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ec2ffd700 1 -- 172.21.15.22:0/2318541686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80608b0 con 0x1be7eb0 2023-06-23T18:04:10.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ec2ffd700 1 -- 172.21.15.22:0/2318541686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8056900 con 0x1be7eb0 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ec2ffd700 1 -- 172.21.15.22:0/2318541686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8057a50 con 0x1be7eb0 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2318541686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 msgr2=0x1ea03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8085670 tx=0x7f0ec8010380 comp rx=0 tx=0).stop 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2318541686 shutdown_connections 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7710 0x1e55c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1ea08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2318541686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7eb0 0x1ea03b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2318541686 >> 172.21.15.22:0/2318541686 conn(0x1e51ab0 msgr2=0x1e5c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2318541686 shutdown_connections 2023-06-23T18:04:10.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2318541686 wait complete. 2023-06-23T18:04:10.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 -- start start 2023-06-23T18:04:10.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.828+0000 7f0ee25c8740 1 Processor -- start 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1e84910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7eb0 0x1e858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d82f20 con 0x1be7710 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d81890 con 0x1be7eb0 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c6cb20 con 0x1be7ae0 2023-06-23T18:04:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50396/0 (socket says 172.21.15.22:50396) 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2970846720 learned_addr learned my addr 172.21.15.22:0/2970846720 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 10 client.0 _ll_register_callbacks cb 0x1d85350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1e84910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7eb0 0x1e858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 msgr2=0x1e84910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1e84910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7eb0 msgr2=0x1e858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7eb0 0x1e858f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 -- 172.21.15.22:0/2970846720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed80af410 con 0x1be7710 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8085720 tx=0x7f0ec8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb994467a852d247 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8057a50 con 0x1be7710 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ec8057bb0 con 0x1be7710 2023-06-23T18:04:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1db9a00 con 0x1be7710 2023-06-23T18:04:10.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec8060420 con 0x1be7710 2023-06-23T18:04:10.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 10 client.4851 fetch_fsmap learned FSMap version 9 2023-06-23T18:04:10.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x1d86670 con 0x1be7710 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1db9a00 con 0x1be7710 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f0ec8069070 con 0x1be7710 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 10 client.4851 fetch_fsmap finished waiting for FSMap version 9 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.832+0000 7f0ee25c8740 10 client.4851 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 20 client.4851 populate_metadata read hostname 'smithi022' 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1c32790 0x1ea08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 4 client.4851 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 --> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1db9720 con 0x1c32790 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1c32790 0x1ea08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0eb7fff700 10 client.4851 ms_handle_connect on v2:172.21.15.22:6835/2486746630 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1c32790 0x1ea08a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac03674f76dc491d server_cookie=b350f14a40e1a429 in_seq=0 out_seq=0 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0eb7fff700 1 -- 172.21.15.22:0/2970846720 <== mds.0 v2:172.21.15.22:6835/2486746630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x1db9720 con 0x1c32790 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0eb7fff700 10 client.4851 handle_command_reply: tid=0 2023-06-23T18:04:10.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 client.4851 shutdown 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 msgr2=0x1e5c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8085720 tx=0x7f0ec8009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 shutdown_connections 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1c32790 0x1ea08a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7eb0 0x1e858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0eb7fff700 0 client.4851 ms_handle_reset on v2:172.21.15.22:6835/2486746630 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7ae0 0x1e84910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/2970846720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5c1a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 >> 172.21.15.22:0/2970846720 conn(0x1e51ab0 msgr2=0x1e832e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 shutdown_connections 2023-06-23T18:04:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 1 -- 172.21.15.22:0/2970846720 wait complete. 2023-06-23T18:04:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.836+0000 7f0ee25c8740 20 client.4851 trim_cache size 0 max 16384 2023-06-23T18:04:10.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.856+0000 7f0ee25c8740 1 Processor -- start 2023-06-23T18:04:10.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.856+0000 7f0ee25c8740 1 -- start start 2023-06-23T18:04:10.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 0x1e7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1e9e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d81890 con 0x1e51ab0 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d90930 con 0x1be7ae0 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dbc390 con 0x1be7710 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50410/0 (socket says 172.21.15.22:50410) 2023-06-23T18:04:10.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 -- 172.21.15.22:0/3139045235 learned_addr learned my addr 172.21.15.22:0/3139045235 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 0x1e7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1e9e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 -- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 msgr2=0x1e7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 0x1e7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 -- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 msgr2=0x1e9e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1e9e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 -- 172.21.15.22:0/3139045235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8070b50 con 0x1e51ab0 2023-06-23T18:04:10.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 0x1e7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:10.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8060380 tx=0x7f0ec8084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faab4bc81d557c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ec37fe700 1 -- 172.21.15.22:0/3139045235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8060420 con 0x1e51ab0 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ec37fe700 1 -- 172.21.15.22:0/3139045235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8010380 con 0x1e51ab0 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ec37fe700 1 -- 172.21.15.22:0/3139045235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8057a50 con 0x1e51ab0 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- 172.21.15.22:0/3139045235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 msgr2=0x1e7ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8060380 tx=0x7f0ec8084000 comp rx=0 tx=0).stop 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- 172.21.15.22:0/3139045235 shutdown_connections 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1e9e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1be7710 0x1e7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/3139045235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1e51ab0 0x1e7ed40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- 172.21.15.22:0/3139045235 >> 172.21.15.22:0/3139045235 conn(0x1e85a60 msgr2=0x1ea88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- 172.21.15.22:0/3139045235 shutdown_connections 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- 172.21.15.22:0/3139045235 wait complete. 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 -- start start 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.860+0000 7f0ee25c8740 1 Processor -- start 2023-06-23T18:04:10.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1eb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e51ab0 0x1efc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d74690 con 0x1be7710 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d63990 con 0x1be7ae0 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d90170 con 0x1e51ab0 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 10 client.0 _ll_register_callbacks cb 0x1d6e540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50426/0 (socket says 172.21.15.22:50426) 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1381533924 learned_addr learned my addr 172.21.15.22:0/1381533924 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1eb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd8ee700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e51ab0 0x1efc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e51ab0 msgr2=0x1efc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e51ab0 0x1efc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 msgr2=0x1eb6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:10.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1eb6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 -- 172.21.15.22:0/1381533924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed80af410 con 0x1be7710 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1eb6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edd0ed700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8054a60 tx=0x7f0ec80549d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a69cb1625a87ef server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8057c40 con 0x1be7710 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ec805e0b0 con 0x1be7710 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e52930 con 0x1be7710 2023-06-23T18:04:10.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec805e270 con 0x1be7710 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 10 client.4857 fetch_fsmap learned FSMap version 9 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x1d6c9e0 con 0x1be7710 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e52930 con 0x1be7710 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f0ec8075030 con 0x1be7710 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 10 client.4857 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-23T18:04:10.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 20 client.4857 populate_metadata read hostname 'smithi022' 2023-06-23T18:04:10.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1be7eb0 0x1eb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:10.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 4 client.4857 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-06-23T18:04:10.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 --> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0ec8056900 con 0x1be7eb0 2023-06-23T18:04:10.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1be7eb0 0x1eb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:10.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 10 client.4857 ms_handle_connect on v2:172.21.15.22:6835/2486746630 2023-06-23T18:04:10.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0edc8ec700 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1be7eb0 0x1eb7a00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f89268b6c821b2 server_cookie=9858ce5f7b48edeb in_seq=0 out_seq=0 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 1 -- 172.21.15.22:0/1381533924 <== mds.0 v2:172.21.15.22:6835/2486746630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x7f0ec8056900 con 0x1be7eb0 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0eb7fff700 10 client.4857 handle_command_reply: tid=0 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 client.4857 shutdown 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 msgr2=0x1e5a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.864+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8054a60 tx=0x7f0ec80549d0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 shutdown_connections 2023-06-23T18:04:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:6835/2486746630,v1:172.21.15.22:6837/2486746630] conn(0x1be7eb0 0x1eb7a00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e51ab0 0x1efc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1be7ae0 0x1eb6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 --2- 172.21.15.22:0/1381533924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1be7710 0x1e5a4b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0eb7fff700 0 client.4857 ms_handle_reset on v2:172.21.15.22:6835/2486746630 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 >> 172.21.15.22:0/1381533924 conn(0x1e85a60 msgr2=0x1e7ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 shutdown_connections 2023-06-23T18:04:10.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 1 -- 172.21.15.22:0/1381533924 wait complete. 2023-06-23T18:04:10.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0ee25c8740 20 client.4857 trim_cache size 0 max 16384 2023-06-23T18:04:10.889 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:04:10.889 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "e4fadccd-0593-4464-bce3-4c0f7c738354", 2023-06-23T18:04:10.889 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-06-23T18:04:10.889 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4263, 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 33.139269026999997, 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 9, 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 24, 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 24, 2023-06-23T18:04:10.890 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 37.516338822000002 2023-06-23T18:04:10.891 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:04:10.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 msgr2=0x7f0ebc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 0x7f0ebc0687a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8071c80 tx=0x7f0ed4009040 comp rx=0 tx=0).stop 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 msgr2=0x7f0ed8069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80cb490 tx=0x7f0ec8004770 comp rx=0 tx=0).stop 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 shutdown_connections 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed8152dc0 0x7f0ed80ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0ebc0662f0 0x7f0ebc0687a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80cb000 0x7f0ed80adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 --2- 172.21.15.22:0/747273948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed80c97c0 0x7f0ed8069fa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:10.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 >> 172.21.15.22:0/747273948 conn(0x7f0ed8069960 msgr2=0x7f0ed806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:10.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 shutdown_connections 2023-06-23T18:04:10.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:10.868+0000 7f0edf351700 1 -- 172.21.15.22:0/747273948 wait complete. 2023-06-23T18:04:10.900 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-06-23T18:04:10.911 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-06-23T18:04:10.912 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-06-23T18:04:10.912 INFO:tasks.ceph_fuse:client.0 config is {} 2023-06-23T18:04:10.912 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:04:10.913 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:10.913 INFO:tasks.ceph_fuse:client.2 config is {} 2023-06-23T18:04:10.913 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:04:10.913 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:10.914 INFO:tasks.ceph_fuse:client.1 config is {} 2023-06-23T18:04:10.914 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:04:10.914 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:10.914 INFO:tasks.ceph_fuse:client.3 config is {} 2023-06-23T18:04:10.914 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:04:10.915 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:10.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:10.915 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-06-23T18:04:10.923 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-06-23T18:04:10.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:10.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-06-23T18:04:10.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:10.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-06-23T18:04:11.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.110 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-06-23T18:04:11.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-06-23T18:04:11.138 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-06-23T18:04:11.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:04:11.140 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-06-23T18:04:11.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:04:11.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 Processor -- start 2023-06-23T18:04:11.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 -- start start 2023-06-23T18:04:11.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 0x7f42340c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4234071a80 0x7f42340cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42340cb0e0 con 0x7f4234071630 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42341591a0 con 0x7f4234071a80 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f423a912700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42340ca4d0 con 0x7f4234158dd0 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f42337fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4234071630 0x7f42340cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57390/0 (socket says 172.21.15.22:57390) 2023-06-23T18:04:11.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f42337fe700 1 -- 172.21.15.22:0/2109227071 learned_addr learned my addr 172.21.15.22:0/2109227071 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:11.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 --2- 172.21.15.22:0/2109227071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 0x7f42340c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4238eaf700 1 --2- 172.21.15.22:0/2109227071 >> v2:172.21.15.169:3300/0 conn(0x7f4234071a80 0x7f42340cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058960570 0 0) 0x7f42340cb0e0 con 0x7f4234071630 2023-06-23T18:04:11.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42200035d0 con 0x7f4234071630 2023-06-23T18:04:11.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833817279 0 0) 0x7f42200035d0 con 0x7f4234071630 2023-06-23T18:04:11.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 -- 172.21.15.22:0/2109227071 >> v2:172.21.15.169:3300/0 conn(0x7f4234071a80 msgr2=0x7f42340cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 --2- 172.21.15.22:0/2109227071 >> v2:172.21.15.169:3300/0 conn(0x7f4234071a80 0x7f42340cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 -- 172.21.15.22:0/2109227071 >> v1:172.21.15.22:6789/0 conn(0x7f4234071630 legacy=0x7f42340cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 -- 172.21.15.22:0/2109227071 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340ca7b0 con 0x7f4234158dd0 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.248+0000 7f4233fff700 1 --2- 172.21.15.22:0/2109227071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 0x7f42340c9e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f421c009850 tx=0x7f421c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4979e4c69d8001c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0019f0 con 0x7f4234158dd0 2023-06-23T18:04:11.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c004030 con 0x7f4234158dd0 2023-06-23T18:04:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f4232ffd700 1 -- 172.21.15.22:0/2109227071 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0043e0 con 0x7f4234158dd0 2023-06-23T18:04:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- 172.21.15.22:0/2109227071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 msgr2=0x7f42340c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 --2- 172.21.15.22:0/2109227071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 0x7f42340c9e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f421c009850 tx=0x7f421c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- 172.21.15.22:0/2109227071 shutdown_connections 2023-06-23T18:04:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 --2- 172.21.15.22:0/2109227071 >> v2:172.21.15.169:3300/0 conn(0x7f4234071a80 0x7f42340cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 --2- 172.21.15.22:0/2109227071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234158dd0 0x7f42340c9e60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- 172.21.15.22:0/2109227071 >> 172.21.15.22:0/2109227071 conn(0x7f423406c5d0 msgr2=0x7f423406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- 172.21.15.22:0/2109227071 shutdown_connections 2023-06-23T18:04:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- 172.21.15.22:0/2109227071 wait complete. 2023-06-23T18:04:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 Processor -- start 2023-06-23T18:04:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.252+0000 7f423a912700 1 -- start start 2023-06-23T18:04:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4234071630 0x7f4234143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234071a80 0x7f42340cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42340c6f80 con 0x7f4234158dd0 2023-06-23T18:04:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42340c6390 con 0x7f4234071630 2023-06-23T18:04:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42340c65e0 con 0x7f4234071a80 2023-06-23T18:04:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50428/0 (socket says 172.21.15.22:50428) 2023-06-23T18:04:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4233fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4234071630 0x7f4234143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 -- 172.21.15.22:0/496844815 learned_addr learned my addr 172.21.15.22:0/496844815 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f42337fe700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234071a80 0x7f42340cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 -- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234071a80 msgr2=0x7f42340cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234071a80 0x7f42340cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 -- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4234071630 msgr2=0x7f4234143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4234071630 0x7f4234143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 -- 172.21.15.22:0/496844815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0094b0 con 0x7f4234158dd0 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4238eaf700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f422400e900 tx=0x7f422400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224013680 con 0x7f4234158dd0 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42240137e0 con 0x7f4234158dd0 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42340c68c0 con 0x7f4234158dd0 2023-06-23T18:04:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224013b70 con 0x7f4234158dd0 2023-06-23T18:04:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.256+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4234143ab0 con 0x7f4234158dd0 2023-06-23T18:04:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42340c70c0 con 0x7f4234158dd0 2023-06-23T18:04:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4224013cd0 con 0x7f4234158dd0 2023-06-23T18:04:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f4230ff9700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 0x7f42140687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4224083e70 con 0x7f4234158dd0 2023-06-23T18:04:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f4233fff700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 0x7f42140687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.260+0000 7f4233fff700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 0x7f42140687f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f421c0090b0 tx=0x7f421c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.268+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f42340c70c0 con 0x7f4234158dd0 2023-06-23T18:04:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.468+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f42340c70c0 con 0x7f4234158dd0 2023-06-23T18:04:11.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.468+0000 7f4230ff9700 1 -- 172.21.15.22:0/496844815 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f42340c70c0 con 0x7f4234158dd0 2023-06-23T18:04:11.483 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-23T18:04:11.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 msgr2=0x7f42140687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 0x7f42140687f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f421c0090b0 tx=0x7f421c008030 comp rx=0 tx=0).stop 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 msgr2=0x7f42340cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f422400e900 tx=0x7f422400ec10 comp rx=0 tx=0).stop 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 shutdown_connections 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4234158dd0 0x7f42340cf800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4234071a80 0x7f42340cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4214066340 0x7f42140687f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 --2- 172.21.15.22:0/496844815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4234071630 0x7f4234143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:11.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 >> 172.21.15.22:0/496844815 conn(0x7f423406c5d0 msgr2=0x7f42340cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:11.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 shutdown_connections 2023-06-23T18:04:11.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:11.472+0000 7f423a912700 1 -- 172.21.15.22:0/496844815 wait complete. 2023-06-23T18:04:11.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:04:11.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:04:11.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-06-23T18:04:11.503 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-23T18:04:11.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:11.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:04:11.504 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:04:11.504 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:04:11.504 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:04:11.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.505 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-06-23T18:04:11.512 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:11.512 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:11.512 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:11.512 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:11.513 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:11.513 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:11.513 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:11.513 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:11.513 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42553sec preferred_lft 42553sec 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:11.514 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:11.515 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:11.515 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:04:11.515 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:c1:01:7a:e8 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:11.515 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:04:11.515 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:11.515 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:04:11.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.516 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:11.516 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:11.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:04:11.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-06-23T18:04:11.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-06-23T18:04:11.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:04:11.517 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:11.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.559 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:04:11.615 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-06-23T18:04:11.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.617 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-06-23T18:04:11.624 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-06-23T18:04:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:04:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-06-23T18:04:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:04:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-06-23T18:04:11.625 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:04:11.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.627 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:11.627 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:11.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-23T18:04:11.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-23T18:04:11.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-23T18:04:11.628 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:11.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.669 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-06-23T18:04:11.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.719 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-06-23T18:04:11.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.727 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:11.728 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:11.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:04:11.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-23T18:04:11.729 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:11.761 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-23T18:04:11.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.762 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:11.763 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:11.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-23T18:04:11.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:04:11.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-23T18:04:11.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-23T18:04:11.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-23T18:04:11.765 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:11.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.871 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:11.872 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:11.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-06-23T18:04:11.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:04:11.873 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:11.903 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-23T18:04:11.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:11.904 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:11.911 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:04:11.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:11.912 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:11.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:11.978 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:11.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:11.994 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:11.995 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:11.996 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:12.003 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:04:12.004 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-23T18:04:12.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:12.023 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:12.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:12.079 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:12.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:12.080 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:12.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2023-06-23T18:04:12.084+0000 7f87449fc640 -1 init, newargv = 0x559a56200330 newargc=15 2023-06-23T18:04:12.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[43247]: starting ceph client 2023-06-23T18:04:12.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[43247]: starting fuse 2023-06-23T18:04:13.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:13.203 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:13.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:13.264 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:13.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:13.265 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:13.272 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:04:13.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-06-23T18:04:13.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:04:13.273 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-06-23T18:04:13.273 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-06-23T18:04:13.273 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-06-23T18:04:13.274 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-06-23T18:04:13.274 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:13.274 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-06-23T18:04:13.274 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-23T18:04:13.275 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-06-23T18:04:13.275 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-23T18:04:13.275 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:13.275 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-06-23T18:04:13.276 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-06-23T18:04:13.276 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-06-23T18:04:13.276 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-06-23T18:04:13.276 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:13.276 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-06-23T18:04:13.277 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:04:13.277 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:04:13.277 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:04:13.277 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-06-23T18:04:13.278 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-06-23T18:04:13.278 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-06-23T18:04:13.278 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:04:13.278 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:13.278 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2023-06-23T18:04:13.279 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-06-23T18:04:13.359 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.43247.asok 2023-06-23T18:04:13.545 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43247.asok 2023-06-23T18:04:13.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:13.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43247.asok status 2023-06-23T18:04:13.692 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:04:13.692 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-06-23T18:04:13.692 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:04:13.692 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:04:13.692 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "43247", 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-06-23T18:04:13.693 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4869, 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4869 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:04:13.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3438545096 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:04:13.695 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3438545096 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4869 192.168.0.1:0/3438545096", 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/3438545096", 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 9, 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 24, 2023-06-23T18:04:13.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:04:13.697 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-06-23T18:04:13.697 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-06-23T18:04:13.697 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:04:13.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:04:13.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 Processor -- start 2023-06-23T18:04:13.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 -- start start 2023-06-23T18:04:13.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe3101603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:13.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe310160900 con 0x7fe3100b1400 2023-06-23T18:04:13.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3100d1730 con 0x7fe3100cca10 2023-06-23T18:04:13.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3172fc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100cd060 con 0x7fe3100b45b0 2023-06-23T18:04:13.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3162fa700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe3100b1400 0x7fe3100d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57396/0 (socket says 172.21.15.22:57396) 2023-06-23T18:04:13.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3162fa700 1 -- 172.21.15.22:0/2621109730 learned_addr learned my addr 172.21.15.22:0/2621109730 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:13.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe316afb700 1 --2- 172.21.15.22:0/2621109730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe3101603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:13.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786889557 0 0) 0x7fe308001490 con 0x7fe3100b1400 2023-06-23T18:04:13.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.792+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2fc0035d0 con 0x7fe3100b1400 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158825607 0 0) 0x7fe3100d1730 con 0x7fe3100cca10 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe308001490 con 0x7fe3100cca10 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe316afb700 1 -- 172.21.15.22:0/2621109730 >> v1:172.21.15.169:6789/0 conn(0x7fe3100cca10 legacy=0x7fe3100b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365527619 0 0) 0x7fe2fc0035d0 con 0x7fe3100b1400 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe316afb700 1 -- 172.21.15.22:0/2621109730 >> v1:172.21.15.22:6789/0 conn(0x7fe3100b1400 legacy=0x7fe3100d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe316afb700 1 -- 172.21.15.22:0/2621109730 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100cd340 con 0x7fe3100b45b0 2023-06-23T18:04:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe316afb700 1 --2- 172.21.15.22:0/2621109730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe3101603c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009880 tx=0x7fe30c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872d89ca60ae8897 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308001b80 con 0x7fe3100b45b0 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c004030 con 0x7fe3100b45b0 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3152f8700 1 -- 172.21.15.22:0/2621109730 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0043a0 con 0x7fe3100b45b0 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- 172.21.15.22:0/2621109730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 msgr2=0x7fe3101603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2621109730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe3101603c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009880 tx=0x7fe30c009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- 172.21.15.22:0/2621109730 shutdown_connections 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2621109730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe3101603c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- 172.21.15.22:0/2621109730 >> 172.21.15.22:0/2621109730 conn(0x7fe310069960 msgr2=0x7fe310069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- 172.21.15.22:0/2621109730 shutdown_connections 2023-06-23T18:04:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- 172.21.15.22:0/2621109730 wait complete. 2023-06-23T18:04:13.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 Processor -- start 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.796+0000 7fe3172fc700 1 -- start start 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe31015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3100cca10 0x7fe31015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015cef0 con 0x7fe3100b1400 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015f220 con 0x7fe3100cca10 2023-06-23T18:04:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015d160 con 0x7fe3100b45b0 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50440/0 (socket says 172.21.15.22:50440) 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 -- 172.21.15.22:0/2148222324 learned_addr learned my addr 172.21.15.22:0/2148222324 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe316afb700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3100cca10 0x7fe31015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe315af9700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe31015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 -- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 msgr2=0x7fe31015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe31015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:13.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 -- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3100cca10 msgr2=0x7fe31015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3100cca10 0x7fe31015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 -- 172.21.15.22:0/2148222324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0094b0 con 0x7fe3100b1400 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3162fa700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe308001ba0 tx=0x7fe30800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800ad90 con 0x7fe3100b1400 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3080070d0 con 0x7fe3100b1400 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31015d440 con 0x7fe3100b1400 2023-06-23T18:04:13.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308007480 con 0x7fe3100b1400 2023-06-23T18:04:13.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31015da00 con 0x7fe3100b1400 2023-06-23T18:04:13.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.800+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31004e4c0 con 0x7fe3100b1400 2023-06-23T18:04:13.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.804+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe3080075e0 con 0x7fe3100b1400 2023-06-23T18:04:13.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.804+0000 7fe306ffd700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 0x7fe2f40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:13.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.804+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe308084300 con 0x7fe3100b1400 2023-06-23T18:04:13.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.804+0000 7fe315af9700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 0x7fe2f40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:13.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.804+0000 7fe315af9700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 0x7fe2f40687a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3101579a0 tx=0x7fe300009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:13.808+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe3080513e0 con 0x7fe3100b1400 2023-06-23T18:04:14.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.008+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe310160900 con 0x7fe3100b1400 2023-06-23T18:04:14.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe306ffd700 1 -- 172.21.15.22:0/2148222324 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe310160900 con 0x7fe3100b1400 2023-06-23T18:04:14.024 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-23T18:04:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 msgr2=0x7fe2f40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 0x7fe2f40687a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3101579a0 tx=0x7fe300009040 comp rx=0 tx=0).stop 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 msgr2=0x7fe310157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe308001ba0 tx=0x7fe30800a340 comp rx=0 tx=0).stop 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 shutdown_connections 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2f40662f0 0x7fe2f40687a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3100cca10 0x7fe31015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3100b45b0 0x7fe31015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 --2- 172.21.15.22:0/2148222324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3100b1400 0x7fe310157460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 >> 172.21.15.22:0/2148222324 conn(0x7fe310069960 msgr2=0x7fe3100d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 shutdown_connections 2023-06-23T18:04:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:14.012+0000 7fe3172fc700 1 -- 172.21.15.22:0/2148222324 wait complete. 2023-06-23T18:04:14.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:04:14.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:04:14.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-06-23T18:04:14.040 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-06-23T18:04:14.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:14.040 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:04:14.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:04:14.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:04:14.041 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-06-23T18:04:14.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.041 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-06-23T18:04:14.049 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:14.050 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:14.050 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:14.050 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.050 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:14.050 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.051 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:14.051 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.051 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:14.051 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.051 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42551sec preferred_lft 42551sec 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:14.052 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.053 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:04:14.053 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:c1:01:7a:e8 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.053 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:04:14.053 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.053 INFO:teuthology.orchestra.run.smithi022.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 2e:51:f1:51:a4:ce brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::a807:d9ff:fe1b:169a/64 scope link 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 2e:51:f1:51:a4:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:04:14.055 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::2c51:f1ff:fe51:a4ce/64 scope link 2023-06-23T18:04:14.055 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.055 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:04:14.106 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-06-23T18:04:14.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.108 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-06-23T18:04:14.114 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:04:14.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.115 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:14.115 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:14.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-23T18:04:14.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-23T18:04:14.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-23T18:04:14.116 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:14.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.154 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-06-23T18:04:14.161 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-23T18:04:14.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.162 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-06-23T18:04:14.210 INFO:teuthology.orchestra.run.smithi022.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-06-23T18:04:14.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.211 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:14.212 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:14.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-23T18:04:14.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-06-23T18:04:14.212 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:14.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-23T18:04:14.258 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:14.258 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:14.258 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:14.258 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.258 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 8a:f5:f0:4a:b6:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::88f5:f0ff:fe4a:b665/64 scope link 2023-06-23T18:04:14.259 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-23T18:04:14.277 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:14.277 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:14.278 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 8a:f5:f0:4a:b6:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:04:14.279 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:04:14.279 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.279 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::88f5:f0ff:fe4a:b665/64 scope link 2023-06-23T18:04:14.279 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:14.280 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-06-23T18:04:14.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.281 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:14.281 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:14.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-06-23T18:04:14.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-23T18:04:14.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-06-23T18:04:14.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-06-23T18:04:14.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-06-23T18:04:14.283 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:14.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.377 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:04:14.377 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:04:14.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.1 up 2023-06-23T18:04:14.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.1 master ceph-brx 2023-06-23T18:04:14.377 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:04:14.401 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-06-23T18:04:14.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:14.402 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:14.409 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-06-23T18:04:14.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:14.410 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:14.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:14.475 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:14.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:14.491 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.492 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:14.542 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:04:14.542 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-06-23T18:04:14.542 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-06-23T18:04:14.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:14.565 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:14.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:14.623 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:14.624 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:14.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:14.625 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:14.630 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:04:14.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:2023-06-23T18:04:14.620+0000 7f1edbf50640 -1 init, newargv = 0x5625357ef970 newargc=15 2023-06-23T18:04:14.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[43387]: starting ceph client 2023-06-23T18:04:14.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[43387]: starting fuse 2023-06-23T18:04:15.732 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:04:15.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:15.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:15.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:15.769 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:04:15.818 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:04:15.818 INFO:teuthology.orchestra.run.smithi022.stdout:53 2023-06-23T18:04:15.818 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-06-23T18:04:15.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:04:15.819 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-06-23T18:04:15.819 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-06-23T18:04:15.819 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-06-23T18:04:15.820 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-06-23T18:04:15.820 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:15.820 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-06-23T18:04:15.820 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-06-23T18:04:15.821 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-06-23T18:04:15.821 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-06-23T18:04:15.821 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:15.821 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-06-23T18:04:15.821 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-06-23T18:04:15.822 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-06-23T18:04:15.822 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-06-23T18:04:15.822 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:15.822 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-06-23T18:04:15.823 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:04:15.823 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:04:15.823 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:04:15.823 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-06-23T18:04:15.824 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-06-23T18:04:15.824 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-06-23T18:04:15.824 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:04:15.824 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:04:15.824 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.2")) 2023-06-23T18:04:15.825 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-06-23T18:04:15.905 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.2.43387.asok 2023-06-23T18:04:16.091 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43387.asok 2023-06-23T18:04:16.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43387.asok status 2023-06-23T18:04:16.196 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:04:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-06-23T18:04:16.197 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:04:16.197 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:04:16.197 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "2", 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "43387", 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-06-23T18:04:16.198 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4878, 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4878 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:16.199 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1646139541 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:16.200 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1646139541 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4878 192.168.0.2:0/1646139541", 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.2:0/1646139541", 2023-06-23T18:04:16.201 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 9, 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 24, 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-06-23T18:04:16.202 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:04:16.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:04:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 Processor -- start 2023-06-23T18:04:16.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 -- start start 2023-06-23T18:04:16.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20fc0c0fe0 0x7f20fc0c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0c18f0 con 0x7f20fc070040 2023-06-23T18:04:16.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc163ec0 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1641a0 con 0x7f20fc160420 2023-06-23T18:04:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fb59e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f20fc070040 0x7f20fc0afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33940/0 (socket says 172.21.15.22:33940) 2023-06-23T18:04:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fb59e700 1 -- 172.21.15.22:0/261543883 learned_addr learned my addr 172.21.15.22:0/261543883 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 --2- 172.21.15.22:0/261543883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fad9d700 1 --2- 172.21.15.22:0/261543883 >> v2:172.21.15.169:3300/0 conn(0x7f20fc0c0fe0 0x7f20fc0c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20f9d9b700 1 -- 172.21.15.22:0/261543883 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811585032 0 0) 0x7f20fc0c18f0 con 0x7f20fc070040 2023-06-23T18:04:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20f9d9b700 1 -- 172.21.15.22:0/261543883 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e40035d0 con 0x7f20fc070040 2023-06-23T18:04:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 -- 172.21.15.22:0/261543883 >> v2:172.21.15.169:3300/0 conn(0x7f20fc0c0fe0 msgr2=0x7f20fc0c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 --2- 172.21.15.22:0/261543883 >> v2:172.21.15.169:3300/0 conn(0x7f20fc0c0fe0 0x7f20fc0c13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 -- 172.21.15.22:0/261543883 >> v1:172.21.15.22:6789/0 conn(0x7f20fc070040 legacy=0x7f20fc0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 -- 172.21.15.22:0/261543883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc164480 con 0x7f20fc160420 2023-06-23T18:04:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20fa59c700 1 --2- 172.21.15.22:0/261543883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc0cbc70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009850 tx=0x7f20ec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1a6c1d44f8d51b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:16.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20f9d9b700 1 -- 172.21.15.22:0/261543883 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec011830 con 0x7f20fc160420 2023-06-23T18:04:16.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20f9d9b700 1 -- 172.21.15.22:0/261543883 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec011990 con 0x7f20fc160420 2023-06-23T18:04:16.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f20f9d9b700 1 -- 172.21.15.22:0/261543883 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec011d00 con 0x7f20fc160420 2023-06-23T18:04:16.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 -- 172.21.15.22:0/261543883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 msgr2=0x7f20fc0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.308+0000 7f210087f700 1 --2- 172.21.15.22:0/261543883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc0cbc70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009850 tx=0x7f20ec009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:16.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- 172.21.15.22:0/261543883 shutdown_connections 2023-06-23T18:04:16.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 --2- 172.21.15.22:0/261543883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc0cbc70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 --2- 172.21.15.22:0/261543883 >> v2:172.21.15.169:3300/0 conn(0x7f20fc0c0fe0 0x7f20fc0c13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- 172.21.15.22:0/261543883 >> 172.21.15.22:0/261543883 conn(0x7f20fc06a5f0 msgr2=0x7f20fc0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:16.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- 172.21.15.22:0/261543883 shutdown_connections 2023-06-23T18:04:16.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- 172.21.15.22:0/261543883 wait complete. 2023-06-23T18:04:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 Processor -- start 2023-06-23T18:04:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- start start 2023-06-23T18:04:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20fc070040 0x7f20fc09b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc09aa30 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0c3490 con 0x7f20fc070040 2023-06-23T18:04:16.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f210087f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1f5e90 con 0x7f20fc160420 2023-06-23T18:04:16.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49092/0 (socket says 172.21.15.22:49092) 2023-06-23T18:04:16.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 -- 172.21.15.22:0/3062959918 learned_addr learned my addr 172.21.15.22:0/3062959918 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:16.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fb59e700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fad9d700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20fc070040 0x7f20fc09b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 -- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 msgr2=0x7f20fc1f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc1f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 -- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20fc070040 msgr2=0x7f20fc09b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20fc070040 0x7f20fc09b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 -- 172.21.15.22:0/3062959918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0094b0 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.312+0000 7f20fa59c700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20ec014040 tx=0x7f20ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:16.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0045d0 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec01e070 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc1f6130 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec003900 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc1f66f0 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc04e4c0 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f20ec003a60 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20e37fe700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 0x7f20e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:16.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.316+0000 7f20fad9d700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 0x7f20e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:16.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.320+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f20ec085190 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.320+0000 7f20fad9d700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 0x7f20e80687a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009b80 tx=0x7f20f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:16.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.328+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f20ec052310 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.528+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20fc0c9020 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.528+0000 7f20e37fe700 1 -- 172.21.15.22:0/3062959918 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f20ec055930 con 0x7f20fc0c0fe0 2023-06-23T18:04:16.543 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 msgr2=0x7f20e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 0x7f20e80687a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009b80 tx=0x7f20f0008040 comp rx=0 tx=0).stop 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 msgr2=0x7f20fc09a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20ec014040 tx=0x7f20ec004770 comp rx=0 tx=0).stop 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 shutdown_connections 2023-06-23T18:04:16.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20fc160420 0x7f20fc1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20fc0c0fe0 0x7f20fc09a4f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f20e80662f0 0x7f20e80687a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 --2- 172.21.15.22:0/3062959918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20fc070040 0x7f20fc09b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:16.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 >> 172.21.15.22:0/3062959918 conn(0x7f20fc06a5f0 msgr2=0x7f20fc06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:16.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 shutdown_connections 2023-06-23T18:04:16.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:16.532+0000 7f210087f700 1 -- 172.21.15.22:0/3062959918 wait complete. 2023-06-23T18:04:16.557 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:04:16.558 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:04:16.558 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-06-23T18:04:16.558 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-23T18:04:16.558 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:16.559 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:04:16.559 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:04:16.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:04:16.559 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:04:16.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.560 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-06-23T18:04:16.567 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:16.568 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:16.568 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:16.568 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:16.569 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42555sec preferred_lft 42555sec 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:16.570 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:04:16.571 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:9f:88:72:e6 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:16.571 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:04:16.571 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:16.571 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:04:16.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.572 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:16.572 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:16.572 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:04:16.572 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2023-06-23T18:04:16.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2023-06-23T18:04:16.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:04:16.573 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:16.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.654 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:04:16.664 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-06-23T18:04:16.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.666 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-06-23T18:04:16.705 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-06-23T18:04:16.705 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:04:16.705 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-06-23T18:04:16.705 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:04:16.706 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:04:16.706 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-06-23T18:04:16.706 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-06-23T18:04:16.706 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:04:16.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.707 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:16.707 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:16.707 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:04:16.707 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:04:16.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:04:16.708 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.750 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-06-23T18:04:16.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.798 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-06-23T18:04:16.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.804 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:16.804 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:16.804 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:04:16.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-06-23T18:04:16.805 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:16.836 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-06-23T18:04:16.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.837 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:16.837 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:16.837 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-06-23T18:04:16.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:04:16.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-06-23T18:04:16.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-06-23T18:04:16.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-06-23T18:04:16.839 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:16.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:16.911 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:16.912 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:16.912 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-06-23T18:04:16.912 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:04:16.912 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:16.942 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-06-23T18:04:16.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:16.942 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:16.967 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-23T18:04:16.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:16.967 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:17.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2023-06-23T18:04:17.033 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:17.033 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:17.044 INFO:teuthology.orchestra.run.smithi169.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:17.045 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:17.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:17.046 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2023-06-23T18:04:17.093 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:04:17.093 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-06-23T18:04:17.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2023-06-23T18:04:17.112 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:17.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:17.122 INFO:teuthology.orchestra.run.smithi169.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:17.123 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:17.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:17.123 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2023-06-23T18:04:17.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:2023-06-23T18:04:17.177+0000 7f7d8ac42640 -1 init, newargv = 0x5631106538a0 newargc=15 2023-06-23T18:04:17.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[41282]: starting ceph client 2023-06-23T18:04:17.193 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[41282]: starting fuse 2023-06-23T18:04:18.231 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2023-06-23T18:04:18.251 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:18.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:18.309 INFO:teuthology.orchestra.run.smithi169.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:18.310 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:18.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:18.311 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2023-06-23T18:04:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:52 2023-06-23T18:04:18.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-06-23T18:04:18.317 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:04:18.317 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2023-06-23T18:04:18.318 DEBUG:teuthology.orchestra.run.smithi169:> import re 2023-06-23T18:04:18.318 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-06-23T18:04:18.318 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-06-23T18:04:18.318 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:18.318 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2023-06-23T18:04:18.319 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-06-23T18:04:18.319 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2023-06-23T18:04:18.319 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-06-23T18:04:18.319 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:18.320 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2023-06-23T18:04:18.320 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2023-06-23T18:04:18.320 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2023-06-23T18:04:18.320 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2023-06-23T18:04:18.321 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:18.321 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2023-06-23T18:04:18.321 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:04:18.321 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:04:18.321 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:04:18.322 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2023-06-23T18:04:18.322 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2023-06-23T18:04:18.322 DEBUG:teuthology.orchestra.run.smithi169:> return f 2023-06-23T18:04:18.322 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:04:18.323 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:18.323 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.1")) 2023-06-23T18:04:18.323 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-06-23T18:04:18.402 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.1.41282.asok 2023-06-23T18:04:18.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41282.asok 2023-06-23T18:04:18.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:18.594 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41282.asok status 2023-06-23T18:04:18.736 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "1", 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2023-06-23T18:04:18.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "41282", 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4688, 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2023-06-23T18:04:18.738 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 4688 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1983330122 2023-06-23T18:04:18.739 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1983330122 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:18.740 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.4688 192.168.0.1:0/1983330122", 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.1:0/1983330122", 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 9, 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 24, 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2023-06-23T18:04:18.741 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2023-06-23T18:04:18.742 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-06-23T18:04:18.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:04:18.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 Processor -- start 2023-06-23T18:04:18.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 -- start start 2023-06-23T18:04:18.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f476c0cf450 0x7f476c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c15c740 con 0x7f476c0b10b0 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0cd500 con 0x7f476c15c370 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476b59e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0d0fc0 con 0x7f476c0cf450 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476ad9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f476c0cf450 0x7f476c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476ad9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f476c0cf450 0x7f476c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49098/0 (socket says 172.21.15.22:49098) 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f476ad9d700 1 -- 172.21.15.22:0/2659216460 learned_addr learned my addr 172.21.15.22:0/2659216460 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:18.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.857+0000 7f4769d9b700 1 --1- 172.21.15.22:0/2659216460 >> v1:172.21.15.22:6789/0 conn(0x7f476c0b10b0 0x7f476c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33944/0 (socket says 172.21.15.22:33944) 2023-06-23T18:04:18.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476ad9d700 1 -- 172.21.15.22:0/2659216460 >> v1:172.21.15.169:6789/0 conn(0x7f476c15c370 legacy=0x7f476c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:18.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476ad9d700 1 -- 172.21.15.22:0/2659216460 >> v1:172.21.15.22:6789/0 conn(0x7f476c0b10b0 legacy=0x7f476c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:18.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476ad9d700 1 -- 172.21.15.22:0/2659216460 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0d0620 con 0x7f476c0cf450 2023-06-23T18:04:18.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476ad9d700 1 --2- 172.21.15.22:0/2659216460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0c5ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4764009e50 tx=0x7f476400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c386d79a05517991 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476959a700 1 -- 172.21.15.22:0/2659216460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400e070 con 0x7f476c0cf450 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476959a700 1 -- 172.21.15.22:0/2659216460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476400b9c0 con 0x7f476c0cf450 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476959a700 1 -- 172.21.15.22:0/2659216460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400bd70 con 0x7f476c0cf450 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- 172.21.15.22:0/2659216460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 msgr2=0x7f476c0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 --2- 172.21.15.22:0/2659216460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0c5ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4764009e50 tx=0x7f476400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- 172.21.15.22:0/2659216460 shutdown_connections 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 --2- 172.21.15.22:0/2659216460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0c5ef0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- 172.21.15.22:0/2659216460 >> 172.21.15.22:0/2659216460 conn(0x7f476c069960 msgr2=0x7f476c069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- 172.21.15.22:0/2659216460 shutdown_connections 2023-06-23T18:04:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- 172.21.15.22:0/2659216460 wait complete. 2023-06-23T18:04:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 Processor -- start 2023-06-23T18:04:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.861+0000 7f476b59e700 1 -- start start 2023-06-23T18:04:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0b10b0 0x7f476c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c15c370 0x7f476c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0cd330 con 0x7f476c0cf450 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0cca90 con 0x7f476c15c370 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0ccd70 con 0x7f476c0b10b0 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49108/0 (socket says 172.21.15.22:49108) 2023-06-23T18:04:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476a59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0b10b0 0x7f476c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 -- 172.21.15.22:0/1670355134 learned_addr learned my addr 172.21.15.22:0/1670355134 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476ad9d700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c15c370 0x7f476c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 -- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0b10b0 msgr2=0x7f476c153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0b10b0 0x7f476c153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 -- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c15c370 msgr2=0x7f476c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c15c370 0x7f476c0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 -- 172.21.15.22:0/1670355134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009b30 con 0x7f476c0cf450 2023-06-23T18:04:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f4769d9b700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c460 tx=0x7f475c00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c005cd0 con 0x7f476c0cf450 2023-06-23T18:04:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c005e30 con 0x7f476c0cf450 2023-06-23T18:04:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0072d0 con 0x7f476c0cf450 2023-06-23T18:04:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0cd050 con 0x7f476c0cf450 2023-06-23T18:04:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.865+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c154350 con 0x7f476c0cf450 2023-06-23T18:04:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c0b5ff0 con 0x7f476c0cf450 2023-06-23T18:04:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f475c007430 con 0x7f476c0cf450 2023-06-23T18:04:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f475affd700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 0x7f474c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f475c083260 con 0x7f476c0cf450 2023-06-23T18:04:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f476a59c700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 0x7f474c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.869+0000 7f476a59c700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 0x7f474c0687d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4760001800 tx=0x7f4760009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:18.877+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f475c050410 con 0x7f476c0cf450 2023-06-23T18:04:18.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:04:18.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:04:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f476c04e4c0 con 0x7f476c0cf450 2023-06-23T18:04:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f475affd700 1 -- 172.21.15.22:0/1670355134 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f475c017070 con 0x7f476c0cf450 2023-06-23T18:04:19.096 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 msgr2=0x7f474c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 0x7f474c0687d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4760001800 tx=0x7f4760009040 comp rx=0 tx=0).stop 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 msgr2=0x7f476c0cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c460 tx=0x7f475c00c770 comp rx=0 tx=0).stop 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 shutdown_connections 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f474c066320 0x7f474c0687d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:19.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f476c15c370 0x7f476c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c0cf450 0x7f476c0cbc10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 --2- 172.21.15.22:0/1670355134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f476c0b10b0 0x7f476c153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 >> 172.21.15.22:0/1670355134 conn(0x7f476c069960 msgr2=0x7f476c0c6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 shutdown_connections 2023-06-23T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:19.085+0000 7f476b59e700 1 -- 172.21.15.22:0/1670355134 wait complete. 2023-06-23T18:04:19.111 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:04:19.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:04:19.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-06-23T18:04:19.111 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-06-23T18:04:19.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:19.112 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:04:19.112 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:04:19.112 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:04:19.112 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-06-23T18:04:19.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.113 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-06-23T18:04:19.121 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:19.121 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:19.121 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:19.121 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.122 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42552sec preferred_lft 42552sec 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-06-23T18:04:19.123 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:9f:88:72:e6 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 9a:22:5a:e6:b0:1b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:04:19.124 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::1c9c:d8ff:fe23:f877/64 scope link 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 9a:22:5a:e6:b0:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::9822:5aff:fee6:b01b/64 scope link 2023-06-23T18:04:19.125 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.126 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:04:19.178 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-06-23T18:04:19.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.180 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-06-23T18:04:19.189 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-06-23T18:04:19.190 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-06-23T18:04:19.191 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:04:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.191 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:19.191 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:19.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-23T18:04:19.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-23T18:04:19.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-23T18:04:19.193 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:19.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.233 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-06-23T18:04:19.281 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-23T18:04:19.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.282 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-06-23T18:04:19.289 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-06-23T18:04:19.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.290 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:19.290 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:19.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-23T18:04:19.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-06-23T18:04:19.291 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:19.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-23T18:04:19.382 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:19.382 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether e2:be:37:4f:38:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:04:19.383 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:04:19.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.384 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::e0be:37ff:fe4f:38b4/64 scope link 2023-06-23T18:04:19.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-23T18:04:19.398 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:04:19.399 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:04:19.399 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:04:19.399 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.400 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-06-23T18:04:19.400 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.400 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:04:19.401 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether e2:be:37:4f:38:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-23T18:04:19.401 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-23T18:04:19.401 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.402 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::e0be:37ff:fe4f:38b4/64 scope link 2023-06-23T18:04:19.402 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:04:19.403 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-06-23T18:04:19.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.404 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:19.405 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:19.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-06-23T18:04:19.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-23T18:04:19.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-06-23T18:04:19.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-06-23T18:04:19.407 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-06-23T18:04:19.407 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:19.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.468 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-06-23T18:04:19.468 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-06-23T18:04:19.469 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2023-06-23T18:04:19.469 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2023-06-23T18:04:19.470 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-06-23T18:04:19.497 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-06-23T18:04:19.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:19.498 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:19.546 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-06-23T18:04:19.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:04:19.547 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:19.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2023-06-23T18:04:19.569 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:19.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:19.626 INFO:teuthology.orchestra.run.smithi169.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:19.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:19.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.628 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2023-06-23T18:04:19.635 INFO:teuthology.orchestra.run.smithi169.stdout:52 2023-06-23T18:04:19.636 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-06-23T18:04:19.636 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-06-23T18:04:19.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2023-06-23T18:04:19.657 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:04:19.658 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:04:19.713 INFO:teuthology.orchestra.run.smithi169.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:04:19.744 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:04:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:19.746 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2023-06-23T18:04:19.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:2023-06-23T18:04:19.721+0000 7fd312890640 -1 init, newargv = 0x5583ea890920 newargc=15 2023-06-23T18:04:19.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:ceph-fuse[41395]: starting ceph client 2023-06-23T18:04:19.748 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:ceph-fuse[41395]: starting fuse 2023-06-23T18:04:19.754 INFO:teuthology.orchestra.run.smithi169.stdout:52 2023-06-23T18:04:19.754 INFO:teuthology.orchestra.run.smithi169.stdout:53 2023-06-23T18:04:19.755 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-06-23T18:04:19.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:04:19.756 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2023-06-23T18:04:19.756 DEBUG:teuthology.orchestra.run.smithi169:> import re 2023-06-23T18:04:19.757 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-06-23T18:04:19.757 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-06-23T18:04:19.757 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:19.758 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2023-06-23T18:04:19.758 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-06-23T18:04:19.759 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2023-06-23T18:04:19.759 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-06-23T18:04:19.760 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:19.760 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2023-06-23T18:04:19.761 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2023-06-23T18:04:19.761 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2023-06-23T18:04:19.762 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2023-06-23T18:04:19.762 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:19.762 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2023-06-23T18:04:19.763 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:04:19.763 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:04:19.764 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:04:19.764 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2023-06-23T18:04:19.765 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2023-06-23T18:04:19.765 DEBUG:teuthology.orchestra.run.smithi169:> return f 2023-06-23T18:04:19.766 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:04:19.766 DEBUG:teuthology.orchestra.run.smithi169:> 2023-06-23T18:04:19.767 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.3")) 2023-06-23T18:04:19.767 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-06-23T18:04:19.848 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.3.41395.asok 2023-06-23T18:04:20.034 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.41395.asok 2023-06-23T18:04:20.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.41395.asok status 2023-06-23T18:04:20.181 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-06-23T18:04:20.181 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "3", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "41395", 2023-06-23T18:04:20.182 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4697, 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2023-06-23T18:04:20.183 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 4697 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 923272480 2023-06-23T18:04:20.184 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.2:0", 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 923272480 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.4697 192.168.0.2:0/923272480", 2023-06-23T18:04:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.2:0/923272480", 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 9, 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 24, 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2023-06-23T18:04:20.186 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-06-23T18:04:20.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.192 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-06-23T18:04:20.192 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:20.200 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-06-23T18:04:20.201 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:20.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:20.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.218 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-06-23T18:04:20.219 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:20.267 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-06-23T18:04:20.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:20.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.285 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2023-06-23T18:04:20.285 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:20.293 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2023-06-23T18:04:20.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:20.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:20.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.311 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2023-06-23T18:04:20.311 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:20.363 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2023-06-23T18:04:20.363 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:20.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:20.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:20.379 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-06-23T18:04:20.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.533+0000 7f3b13fff700 1 Processor -- start 2023-06-23T18:04:20.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.533+0000 7f3b13fff700 1 -- start start 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b14157f00 0x7f3b140cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b140cd160 0x7f3b140cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b1415baf0 0x7f3b1415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140cdaf0 con 0x7f3b1415baf0 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140c9780 con 0x7f3b14157f00 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b13fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140c4c50 con 0x7f3b140cd160 2023-06-23T18:04:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b1415baf0 0x7f3b1415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b1415baf0 0x7f3b1415bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49122/0 (socket says 172.21.15.22:49122) 2023-06-23T18:04:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 -- 172.21.15.22:0/2310623509 learned_addr learned my addr 172.21.15.22:0/2310623509 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b12ffd700 1 --2- 172.21.15.22:0/2310623509 >> v2:172.21.15.169:3300/0 conn(0x7f3b14157f00 0x7f3b140cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b0bfff700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b140cd160 0x7f3b140cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 -- 172.21.15.22:0/2310623509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b140cd160 msgr2=0x7f3b140cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b140cd160 0x7f3b140cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 -- 172.21.15.22:0/2310623509 >> v2:172.21.15.169:3300/0 conn(0x7f3b14157f00 msgr2=0x7f3b140cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 --2- 172.21.15.22:0/2310623509 >> v2:172.21.15.169:3300/0 conn(0x7f3b14157f00 0x7f3b140cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 -- 172.21.15.22:0/2310623509 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140c4f30 con 0x7f3b1415baf0 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b137fe700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b1415baf0 0x7f3b1415bf70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612582e246d4f2d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b127fc700 1 -- 172.21.15.22:0/2310623509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00e070 con 0x7f3b1415baf0 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b127fc700 1 -- 172.21.15.22:0/2310623509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c00b8d0 con 0x7f3b1415baf0 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.537+0000 7f3b127fc700 1 -- 172.21.15.22:0/2310623509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00bc80 con 0x7f3b1415baf0 2023-06-23T18:04:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- 172.21.15.22:0/2310623509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b1415baf0 msgr2=0x7f3b1415bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b1415baf0 0x7f3b1415bf70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- 172.21.15.22:0/2310623509 shutdown_connections 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b1415baf0 0x7f3b1415bf70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- 172.21.15.22:0/2310623509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b140cd160 0x7f3b140cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- 172.21.15.22:0/2310623509 >> v2:172.21.15.169:3300/0 conn(0x7f3b14157f00 0x7f3b140cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- 172.21.15.22:0/2310623509 >> 172.21.15.22:0/2310623509 conn(0x7f3b140cb2a0 msgr2=0x7f3b140cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- 172.21.15.22:0/2310623509 shutdown_connections 2023-06-23T18:04:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- 172.21.15.22:0/2310623509 wait complete. 2023-06-23T18:04:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 Processor -- start 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- start start 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b14157f00 0x7f3b1414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 0x7f3b140b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14155590 con 0x7f3b140cd160 2023-06-23T18:04:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14155870 con 0x7f3b1415baf0 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b13fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14155b50 con 0x7f3b14157f00 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b12ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b12ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49136/0 (socket says 172.21.15.22:49136) 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b12ffd700 1 -- 172.21.15.22:0/3272782378 learned_addr learned my addr 172.21.15.22:0/3272782378 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b0bfff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b14157f00 0x7f3b1414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.541+0000 7f3b137fe700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 0x7f3b140b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 -- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b14157f00 msgr2=0x7f3b1414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b14157f00 0x7f3b1414e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 -- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 msgr2=0x7f3b140b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 0x7f3b140b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 -- 172.21.15.22:0/3272782378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c009b30 con 0x7f3b140cd160 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b137fe700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 0x7f3b140b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b12ffd700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9d0 tx=0x7f3b0400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04009720 con 0x7f3b140cd160 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0400c850 con 0x7f3b140cd160 2023-06-23T18:04:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b14155e30 con 0x7f3b140cd160 2023-06-23T18:04:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400cbc0 con 0x7f3b140cd160 2023-06-23T18:04:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b140b4490 con 0x7f3b140cd160 2023-06-23T18:04:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1404e4c0 con 0x7f3b140cd160 2023-06-23T18:04:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.545+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3b0400cd20 con 0x7f3b140cd160 2023-06-23T18:04:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.549+0000 7f3b0b7fe700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 0x7f3af40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.549+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3b04083fc0 con 0x7f3b140cd160 2023-06-23T18:04:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.549+0000 7f3b0bfff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 0x7f3af40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.549+0000 7f3b0bfff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 0x7f3af40687d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3b1415c620 tx=0x7f3afc009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.553+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3b040b4700 con 0x7f3b140cd160 2023-06-23T18:04:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.721+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b141556d0 con 0x7f3b140cd160 2023-06-23T18:04:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.725+0000 7f3b0b7fe700 1 -- 172.21.15.22:0/3272782378 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f3b141556d0 con 0x7f3b140cd160 2023-06-23T18:04:20.737 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:20.737 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.725+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 msgr2=0x7f3af40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.725+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 0x7f3af40687d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3b1415c620 tx=0x7f3afc009240 comp rx=0 tx=0).stop 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.725+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 msgr2=0x7f3b1414d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.725+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9d0 tx=0x7f3b0400ece0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 shutdown_connections 2023-06-23T18:04:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3af4066320 0x7f3af40687d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1415baf0 0x7f3b140b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b14157f00 0x7f3b1414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 --2- 172.21.15.22:0/3272782378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b140cd160 0x7f3b1414d280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 >> 172.21.15.22:0/3272782378 conn(0x7f3b140cb2a0 msgr2=0x7f3b1415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 shutdown_connections 2023-06-23T18:04:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.729+0000 7f3b13fff700 1 -- 172.21.15.22:0/3272782378 wait complete. 2023-06-23T18:04:20.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:20.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:20.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 Processor -- start 2023-06-23T18:04:20.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 -- start start 2023-06-23T18:04:20.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f340c86c0 0x7f2f340ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f3406cfc0 0x7f2f34159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f340cd7a0 con 0x7f2f3406da30 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f34071080 con 0x7f2f3406cfc0 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f34071710 con 0x7f2f340c86c0 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f31d9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f340c86c0 0x7f2f340ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3259c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2f3406da30 0x7f2f340acdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:45810/0 (socket says 172.21.15.22:45810) 2023-06-23T18:04:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f3259c700 1 -- 172.21.15.22:0/1036227845 learned_addr learned my addr 172.21.15.22:0/1036227845 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:20.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.865+0000 7f2f32d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f3406cfc0 0x7f2f34159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f31d9b700 1 -- 172.21.15.22:0/1036227845 >> v2:172.21.15.22:3300/0 conn(0x7f2f3406cfc0 msgr2=0x7f2f34159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:20.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f31d9b700 1 --2- 172.21.15.22:0/1036227845 >> v2:172.21.15.22:3300/0 conn(0x7f2f3406cfc0 0x7f2f34159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f31d9b700 1 -- 172.21.15.22:0/1036227845 >> v1:172.21.15.169:6789/0 conn(0x7f2f3406da30 legacy=0x7f2f340acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f31d9b700 1 -- 172.21.15.22:0/1036227845 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340cd9f0 con 0x7f2f340c86c0 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f31d9b700 1 --2- 172.21.15.22:0/1036227845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f340c86c0 0x7f2f340ad4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f3406cb40 tx=0x7f2f2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a682f0c1a30f546c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3159a700 1 -- 172.21.15.22:0/1036227845 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e070 con 0x7f2f340c86c0 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3159a700 1 -- 172.21.15.22:0/1036227845 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2400bb20 con 0x7f2f340c86c0 2023-06-23T18:04:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3159a700 1 -- 172.21.15.22:0/1036227845 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24012430 con 0x7f2f340c86c0 2023-06-23T18:04:20.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 -- 172.21.15.22:0/1036227845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f340c86c0 msgr2=0x7f2f340ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 --2- 172.21.15.22:0/1036227845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f340c86c0 0x7f2f340ad4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f3406cb40 tx=0x7f2f2400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:20.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 -- 172.21.15.22:0/1036227845 shutdown_connections 2023-06-23T18:04:20.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 --2- 172.21.15.22:0/1036227845 >> v2:172.21.15.22:3300/0 conn(0x7f2f3406cfc0 0x7f2f34159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 --2- 172.21.15.22:0/1036227845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f340c86c0 0x7f2f340ad4d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 -- 172.21.15.22:0/1036227845 >> 172.21.15.22:0/1036227845 conn(0x7f2f3406a5f0 msgr2=0x7f2f340caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 -- 172.21.15.22:0/1036227845 shutdown_connections 2023-06-23T18:04:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.869+0000 7f2f3359e700 1 -- 172.21.15.22:0/1036227845 wait complete. 2023-06-23T18:04:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 Processor -- start 2023-06-23T18:04:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 -- start start 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f3406cfc0 0x7f2f34150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f3406da30 0x7f2f3406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34099180 con 0x7f2f340c86c0 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34151860 con 0x7f2f3406cfc0 2023-06-23T18:04:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.873+0000 7f2f3359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340c48b0 con 0x7f2f3406da30 2023-06-23T18:04:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49154/0 (socket says 172.21.15.22:49154) 2023-06-23T18:04:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 -- 172.21.15.22:0/4151267632 learned_addr learned my addr 172.21.15.22:0/4151267632 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f3259c700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f3406cfc0 0x7f2f34150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f31d9b700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f3406da30 0x7f2f3406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 -- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f3406da30 msgr2=0x7f2f3406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f3406da30 0x7f2f3406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 -- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f3406cfc0 msgr2=0x7f2f34150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f3406cfc0 0x7f2f34150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 -- 172.21.15.22:0/4151267632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009a70 con 0x7f2f340c86c0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f32d9d700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e7f0 tx=0x7f2f2c00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0095c0 con 0x7f2f340c86c0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c013680 con 0x7f2f340c86c0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f34099390 con 0x7f2f340c86c0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c013a10 con 0x7f2f340c86c0 2023-06-23T18:04:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34099950 con 0x7f2f340c86c0 2023-06-23T18:04:20.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.877+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3404e4c0 con 0x7f2f340c86c0 2023-06-23T18:04:20.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.881+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2f2c013b90 con 0x7f2f340c86c0 2023-06-23T18:04:20.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.881+0000 7f2f2affd700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 0x7f2f140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:20.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.881+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f2c083e40 con 0x7f2f340c86c0 2023-06-23T18:04:20.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.881+0000 7f2f3259c700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 0x7f2f140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:20.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.881+0000 7f2f3259c700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 0x7f2f140687a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f34070970 tx=0x7f2f1c007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:20.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:20.889+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2f2c054a60 con 0x7f2f340c86c0 2023-06-23T18:04:21.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.053+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f34158b20 con 0x7f2f340c86c0 2023-06-23T18:04:21.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.053+0000 7f2f2affd700 1 -- 172.21.15.22:0/4151267632 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f2c0545f0 con 0x7f2f340c86c0 2023-06-23T18:04:21.066 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:21.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 msgr2=0x7f2f140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 0x7f2f140687a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f34070970 tx=0x7f2f1c007f30 comp rx=0 tx=0).stop 2023-06-23T18:04:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 msgr2=0x7f2f34070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e7f0 tx=0x7f2f2c00eb00 comp rx=0 tx=0).stop 2023-06-23T18:04:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 shutdown_connections 2023-06-23T18:04:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f340c86c0 0x7f2f34070200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f3406da30 0x7f2f3406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f140662f0 0x7f2f140687a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 --2- 172.21.15.22:0/4151267632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f3406cfc0 0x7f2f34150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 >> 172.21.15.22:0/4151267632 conn(0x7f2f3406a5f0 msgr2=0x7f2f340ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 shutdown_connections 2023-06-23T18:04:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.057+0000 7f2f3359e700 1 -- 172.21.15.22:0/4151267632 wait complete. 2023-06-23T18:04:21.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:21.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.197+0000 7fea6e8ac700 1 Processor -- start 2023-06-23T18:04:21.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.197+0000 7fea6e8ac700 1 -- start start 2023-06-23T18:04:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea68069960 0x7fea68069d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea680707b0 0x7fea6815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6806a270 con 0x7fea68070c10 2023-06-23T18:04:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea68071110 con 0x7fea680707b0 2023-06-23T18:04:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6815c310 con 0x7fea68069960 2023-06-23T18:04:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea680707b0 0x7fea6815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea680707b0 0x7fea6815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49158/0 (socket says 172.21.15.22:49158) 2023-06-23T18:04:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 -- 172.21.15.22:0/3594663518 learned_addr learned my addr 172.21.15.22:0/3594663518 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d8aa700 1 --2- 172.21.15.22:0/3594663518 >> v2:172.21.15.169:3300/0 conn(0x7fea68069960 0x7fea68069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6c8a8700 1 -- 172.21.15.22:0/3594663518 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890434037 0 0) 0x7fea6806a270 con 0x7fea68070c10 2023-06-23T18:04:21.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6c8a8700 1 -- 172.21.15.22:0/3594663518 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea540035d0 con 0x7fea68070c10 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 -- 172.21.15.22:0/3594663518 >> v2:172.21.15.169:3300/0 conn(0x7fea68069960 msgr2=0x7fea68069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 --2- 172.21.15.22:0/3594663518 >> v2:172.21.15.169:3300/0 conn(0x7fea68069960 0x7fea68069d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 -- 172.21.15.22:0/3594663518 >> v1:172.21.15.169:6789/0 conn(0x7fea68070c10 legacy=0x7fea6815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 -- 172.21.15.22:0/3594663518 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea680ca720 con 0x7fea680707b0 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6d0a9700 1 --2- 172.21.15.22:0/3594663518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea680707b0 0x7fea6815f990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea68071dd0 tx=0x7fea58009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e207489f1050081 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6c8a8700 1 -- 172.21.15.22:0/3594663518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58011830 con 0x7fea680707b0 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6c8a8700 1 -- 172.21.15.22:0/3594663518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58011990 con 0x7fea680707b0 2023-06-23T18:04:21.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6c8a8700 1 -- 172.21.15.22:0/3594663518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58011d40 con 0x7fea680707b0 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3594663518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea680707b0 msgr2=0x7fea6815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3594663518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea680707b0 0x7fea6815f990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea68071dd0 tx=0x7fea58009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3594663518 shutdown_connections 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3594663518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea680707b0 0x7fea6815f990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3594663518 >> v2:172.21.15.169:3300/0 conn(0x7fea68069960 0x7fea68069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3594663518 >> 172.21.15.22:0/3594663518 conn(0x7fea6806b700 msgr2=0x7fea680b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3594663518 shutdown_connections 2023-06-23T18:04:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.201+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3594663518 wait complete. 2023-06-23T18:04:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 Processor -- start 2023-06-23T18:04:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 -- start start 2023-06-23T18:04:21.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea68070c10 0x7fea680ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea6815f0c0 0x7fea680ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6815ec30 con 0x7fea68070c10 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6815e020 con 0x7fea68069960 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e8ac700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6815e300 con 0x7fea6815f0c0 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6d0a9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea68070c10 0x7fea680ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6d8aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6d8aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46526/0 (socket says 172.21.15.22:46526) 2023-06-23T18:04:21.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6d8aa700 1 -- 172.21.15.22:0/3508682820 learned_addr learned my addr 172.21.15.22:0/3508682820 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:21.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.205+0000 7fea6e0ab700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea6815f0c0 0x7fea680ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 -- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea6815f0c0 msgr2=0x7fea680ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea6815f0c0 0x7fea680ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 -- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea68070c10 msgr2=0x7fea680ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea68070c10 0x7fea680ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 -- 172.21.15.22:0/3508682820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009570 con 0x7fea68069960 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6d8aa700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea6400e840 tx=0x7fea6400eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640095a0 con 0x7fea68069960 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64013680 con 0x7fea68069960 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6815e5e0 con 0x7fea68069960 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640139f0 con 0x7fea68069960 2023-06-23T18:04:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea681570f0 con 0x7fea68069960 2023-06-23T18:04:21.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.209+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6804e4c0 con 0x7fea68069960 2023-06-23T18:04:21.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.213+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fea64013b50 con 0x7fea68069960 2023-06-23T18:04:21.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.213+0000 7fea5e7fc700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 0x7fea4c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.213+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fea64083e60 con 0x7fea68069960 2023-06-23T18:04:21.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.213+0000 7fea6d0a9700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 0x7fea4c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.213+0000 7fea6d0a9700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 0x7fea4c0687a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea58008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.221+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fea64054a90 con 0x7fea68069960 2023-06-23T18:04:21.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.385+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea6806ef00 con 0x7fea68069960 2023-06-23T18:04:21.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.389+0000 7fea5e7fc700 1 -- 172.21.15.22:0/3508682820 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea64054620 con 0x7fea68069960 2023-06-23T18:04:21.403 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:21.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 msgr2=0x7fea4c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 0x7fea4c0687a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea58008030 comp rx=0 tx=0).stop 2023-06-23T18:04:21.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 msgr2=0x7fea6806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea6400e840 tx=0x7fea6400eb50 comp rx=0 tx=0).stop 2023-06-23T18:04:21.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 shutdown_connections 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea6815f0c0 0x7fea680ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea68070c10 0x7fea680ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fea4c0662f0 0x7fea4c0687a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 --2- 172.21.15.22:0/3508682820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea68069960 0x7fea6806dfd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 >> 172.21.15.22:0/3508682820 conn(0x7fea6806b700 msgr2=0x7fea680aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 shutdown_connections 2023-06-23T18:04:21.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.393+0000 7fea6e8ac700 1 -- 172.21.15.22:0/3508682820 wait complete. 2023-06-23T18:04:21.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:21.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 Processor -- start 2023-06-23T18:04:21.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 -- start start 2023-06-23T18:04:21.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a1015cf40 0x7f5a100cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1006b210 con 0x7f5a1015c630 2023-06-23T18:04:21.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a100cb7f0 con 0x7f5a1015cf40 2023-06-23T18:04:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a14800700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5a1015c630 0x7f5a10158c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33946/0 (socket says 172.21.15.22:33946) 2023-06-23T18:04:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15802700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100cbad0 con 0x7f5a100cb210 2023-06-23T18:04:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a14800700 1 -- 172.21.15.22:0/281864163 learned_addr learned my addr 172.21.15.22:0/281864163 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a15001700 1 --2- 172.21.15.22:0/281864163 >> v2:172.21.15.22:3300/0 conn(0x7f5a1015cf40 0x7f5a100cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.525+0000 7f5a0ffff700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15001700 1 -- 172.21.15.22:0/281864163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 msgr2=0x7f5a1015ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15001700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1015ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15001700 1 -- 172.21.15.22:0/281864163 >> v1:172.21.15.22:6789/0 conn(0x7f5a1015c630 legacy=0x7f5a10158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15001700 1 -- 172.21.15.22:0/281864163 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100cbdb0 con 0x7f5a1015cf40 2023-06-23T18:04:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15001700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a100cc890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d90 tx=0x7f5a0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea72a33feeb0b31f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a0f7fe700 1 -- 172.21.15.22:0/281864163 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800e070 con 0x7f5a1015cf40 2023-06-23T18:04:21.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a0f7fe700 1 -- 172.21.15.22:0/281864163 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0800bbd0 con 0x7f5a1015cf40 2023-06-23T18:04:21.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a0f7fe700 1 -- 172.21.15.22:0/281864163 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08012430 con 0x7f5a1015cf40 2023-06-23T18:04:21.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 -- 172.21.15.22:0/281864163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 msgr2=0x7f5a100cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a100cc890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d90 tx=0x7f5a0800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 -- 172.21.15.22:0/281864163 shutdown_connections 2023-06-23T18:04:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a100cc890 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 --2- 172.21.15.22:0/281864163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 -- 172.21.15.22:0/281864163 >> 172.21.15.22:0/281864163 conn(0x7f5a1006bdd0 msgr2=0x7f5a1006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 -- 172.21.15.22:0/281864163 shutdown_connections 2023-06-23T18:04:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.529+0000 7f5a15802700 1 -- 172.21.15.22:0/281864163 wait complete. 2023-06-23T18:04:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 Processor -- start 2023-06-23T18:04:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- start start 2023-06-23T18:04:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1015c630 0x7f5a1014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014cd80 con 0x7f5a1015cf40 2023-06-23T18:04:21.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014c8f0 con 0x7f5a1015c630 2023-06-23T18:04:21.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014ab90 con 0x7f5a100cb210 2023-06-23T18:04:21.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49190/0 (socket says 172.21.15.22:49190) 2023-06-23T18:04:21.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 -- 172.21.15.22:0/1977449658 learned_addr learned my addr 172.21.15.22:0/1977449658 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:21.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a14800700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a0ffff700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1015c630 0x7f5a1014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 -- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 msgr2=0x7f5a1014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 -- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1015c630 msgr2=0x7f5a1014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1015c630 0x7f5a1014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 -- 172.21.15.22:0/1977449658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08009a70 con 0x7f5a1015cf40 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a14800700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15001700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a08014040 tx=0x7f5a08004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08012430 con 0x7f5a1015cf40 2023-06-23T18:04:21.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08020070 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004280 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1014bce0 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.533+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1014c2a0 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1014caa0 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5a080043e0 con 0x7f5a1015cf40 2023-06-23T18:04:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a0d7fa700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 0x7f59fc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a08022070 con 0x7f5a1015cf40 2023-06-23T18:04:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a14800700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 0x7f59fc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.537+0000 7f5a14800700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 0x7f59fc0687d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000f60 tx=0x7f5a00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.545+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5a1014caa0 con 0x7f5a1015cf40 2023-06-23T18:04:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.713+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a1014ca60 con 0x7f5a1015cf40 2023-06-23T18:04:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.713+0000 7f5a0d7fa700 1 -- 172.21.15.22:0/1977449658 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a08052f60 con 0x7f5a1015cf40 2023-06-23T18:04:21.724 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 msgr2=0x7f59fc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 0x7f59fc0687d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000f60 tx=0x7f5a00009040 comp rx=0 tx=0).stop 2023-06-23T18:04:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 msgr2=0x7f5a1014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a08014040 tx=0x7f5a08004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 shutdown_connections 2023-06-23T18:04:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a1015cf40 0x7f5a1014d8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f59fc066320 0x7f59fc0687d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1015c630 0x7f5a1014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 --2- 172.21.15.22:0/1977449658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a100cb210 0x7f5a1014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 >> 172.21.15.22:0/1977449658 conn(0x7f5a1006bdd0 msgr2=0x7f5a100ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 shutdown_connections 2023-06-23T18:04:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.717+0000 7f5a15802700 1 -- 172.21.15.22:0/1977449658 wait complete. 2023-06-23T18:04:21.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:21.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 Processor -- start 2023-06-23T18:04:21.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 -- start start 2023-06-23T18:04:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40d1f30 0x7ff4a40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff4a40d2360 0x7ff4a40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40ccb30 con 0x7ff4a40cf3a0 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a40d2880 con 0x7ff4a40d2360 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aaff8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40d2b60 con 0x7ff4a40d1f30 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a9ff6700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff4a40cf3a0 0x7ff4a40cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33956/0 (socket says 172.21.15.22:33956) 2023-06-23T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a9ff6700 1 -- 172.21.15.22:0/2342592029 learned_addr learned my addr 172.21.15.22:0/2342592029 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 --2- 172.21.15.22:0/2342592029 >> v2:172.21.15.22:3300/0 conn(0x7ff4a40d2360 0x7ff4a40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a97f5700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40d1f30 0x7ff4a40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a8ff4700 1 -- 172.21.15.22:0/2342592029 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956355578 0 0) 0x7ff4a40ccb30 con 0x7ff4a40cf3a0 2023-06-23T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a8ff4700 1 -- 172.21.15.22:0/2342592029 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4900035d0 con 0x7ff4a40cf3a0 2023-06-23T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 -- 172.21.15.22:0/2342592029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40d1f30 msgr2=0x7ff4a40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40d1f30 0x7ff4a40d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 -- 172.21.15.22:0/2342592029 >> v1:172.21.15.22:6789/0 conn(0x7ff4a40cf3a0 legacy=0x7ff4a40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 -- 172.21.15.22:0/2342592029 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40d2e40 con 0x7ff4a40d2360 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4aa7f7700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d2360 0x7ff4a40d0560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009d90 tx=0x7ff49c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5936ded8de5831 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a8ff4700 1 -- 172.21.15.22:0/2342592029 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00e070 con 0x7ff4a40d2360 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a8ff4700 1 -- 172.21.15.22:0/2342592029 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00bb20 con 0x7ff4a40d2360 2023-06-23T18:04:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.853+0000 7ff4a8ff4700 1 -- 172.21.15.22:0/2342592029 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00be90 con 0x7ff4a40d2360 2023-06-23T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2342592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d2360 msgr2=0x7ff4a40d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d2360 0x7ff4a40d0560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009d90 tx=0x7ff49c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2342592029 shutdown_connections 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d2360 0x7ff4a40d0560 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2342592029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40d1f30 0x7ff4a40d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2342592029 >> 172.21.15.22:0/2342592029 conn(0x7ff4a406a280 msgr2=0x7ff4a409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2342592029 shutdown_connections 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2342592029 wait complete. 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 Processor -- start 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- start start 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40cf3a0 0x7ff4a40b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a40d2360 0x7ff4a414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a415c9d0 con 0x7ff4a40d1f30 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a414ef10 con 0x7ff4a40d2360 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4aaff8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40b6e10 con 0x7ff4a40cf3a0 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.857+0000 7ff4a97f5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49208/0 (socket says 172.21.15.22:49208) 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 -- 172.21.15.22:0/2717827634 learned_addr learned my addr 172.21.15.22:0/2717827634 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a9ff6700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40cf3a0 0x7ff4a40b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4aa7f7700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a40d2360 0x7ff4a414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 -- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40cf3a0 msgr2=0x7ff4a40b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40cf3a0 0x7ff4a40b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 -- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a40d2360 msgr2=0x7ff4a414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a40d2360 0x7ff4a414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 -- 172.21.15.22:0/2717827634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c009a70 con 0x7ff4a40d1f30 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4a97f5700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9d0 tx=0x7ff49400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494009720 con 0x7ff4a40d1f30 2023-06-23T18:04:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400ee70 con 0x7ff4a40d1f30 2023-06-23T18:04:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400c9f0 con 0x7ff4a40d1f30 2023-06-23T18:04:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a40b70f0 con 0x7ff4a40d1f30 2023-06-23T18:04:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40b76b0 con 0x7ff4a40d1f30 2023-06-23T18:04:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.861+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a404e4c0 con 0x7ff4a40d1f30 2023-06-23T18:04:21.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.865+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff49400cb50 con 0x7ff4a40d1f30 2023-06-23T18:04:21.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.865+0000 7ff49a7fc700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 0x7ff4880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:21.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.865+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff494083f20 con 0x7ff4a40d1f30 2023-06-23T18:04:21.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.865+0000 7ff4a9ff6700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 0x7ff4880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:21.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.865+0000 7ff4a9ff6700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 0x7ff4880687a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0001360 tx=0x7ff4a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:21.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:21.873+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff494054bd0 con 0x7ff4a40d1f30 2023-06-23T18:04:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.033+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4a40b18a0 con 0x7ff4a40d1f30 2023-06-23T18:04:22.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff49a7fc700 1 -- 172.21.15.22:0/2717827634 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4940170a0 con 0x7ff4a40d1f30 2023-06-23T18:04:22.047 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 msgr2=0x7ff4880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 0x7ff4880687a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0001360 tx=0x7ff4a0009040 comp rx=0 tx=0).stop 2023-06-23T18:04:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 msgr2=0x7ff4a415c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9d0 tx=0x7ff49400ece0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 shutdown_connections 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff4880662f0 0x7ff4880687a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a40d2360 0x7ff4a414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4a40d1f30 0x7ff4a415c490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 --2- 172.21.15.22:0/2717827634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a40cf3a0 0x7ff4a40b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 >> 172.21.15.22:0/2717827634 conn(0x7ff4a406a280 msgr2=0x7ff4a40b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 shutdown_connections 2023-06-23T18:04:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.037+0000 7ff4aaff8700 1 -- 172.21.15.22:0/2717827634 wait complete. 2023-06-23T18:04:22.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 Processor -- start 2023-06-23T18:04:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 -- start start 2023-06-23T18:04:22.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ad00d0730 0x7f1ad0166c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad00ccb30 con 0x7f1ad00d1f30 2023-06-23T18:04:22.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad00d2880 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad6089700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00d2b60 con 0x7f1ad00d0730 2023-06-23T18:04:22.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad5888700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ad00d0730 0x7f1ad0166c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad4886700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1ad00d1f30 0x7f1ad00d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33962/0 (socket says 172.21.15.22:33962) 2023-06-23T18:04:22.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad5888700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ad00d0730 0x7f1ad0166c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49216/0 (socket says 172.21.15.22:49216) 2023-06-23T18:04:22.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.177+0000 7f1ad4886700 1 -- 172.21.15.22:0/1472391372 learned_addr learned my addr 172.21.15.22:0/1472391372 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:22.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad5888700 1 -- 172.21.15.22:0/1472391372 >> v1:172.21.15.169:6789/0 conn(0x7f1ad00cf3a0 legacy=0x7f1ad00cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad5888700 1 -- 172.21.15.22:0/1472391372 >> v1:172.21.15.22:6789/0 conn(0x7f1ad00d1f30 legacy=0x7f1ad00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad5888700 1 -- 172.21.15.22:0/1472391372 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00d2e40 con 0x7f1ad00d0730 2023-06-23T18:04:22.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad5888700 1 --2- 172.21.15.22:0/1472391372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00d0730 0x7f1ad0166c30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009d90 tx=0x7f1ac800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e5dd5c0cb9cdab server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ac7fff700 1 -- 172.21.15.22:0/1472391372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800e070 con 0x7f1ad00d0730 2023-06-23T18:04:22.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ac7fff700 1 -- 172.21.15.22:0/1472391372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac800bbd0 con 0x7f1ad00d0730 2023-06-23T18:04:22.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ac7fff700 1 -- 172.21.15.22:0/1472391372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8012430 con 0x7f1ad00d0730 2023-06-23T18:04:22.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- 172.21.15.22:0/1472391372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00d0730 msgr2=0x7f1ad0166c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1472391372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00d0730 0x7f1ad0166c30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009d90 tx=0x7f1ac800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- 172.21.15.22:0/1472391372 shutdown_connections 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1472391372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00d0730 0x7f1ad0166c30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- 172.21.15.22:0/1472391372 >> 172.21.15.22:0/1472391372 conn(0x7f1ad006a280 msgr2=0x7f1ad009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- 172.21.15.22:0/1472391372 shutdown_connections 2023-06-23T18:04:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- 172.21.15.22:0/1472391372 wait complete. 2023-06-23T18:04:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 Processor -- start 2023-06-23T18:04:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.181+0000 7f1ad6089700 1 -- start start 2023-06-23T18:04:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 0x7f1ad015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 0x7f1ad0155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad015d900 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00b5d50 con 0x7f1ad00d1f30 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad0156620 con 0x7f1ad00d0730 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49220/0 (socket says 172.21.15.22:49220) 2023-06-23T18:04:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 -- 172.21.15.22:0/1558189406 learned_addr learned my addr 172.21.15.22:0/1558189406 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5888700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 0x7f1ad0155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad4886700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 0x7f1ad015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 -- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 msgr2=0x7f1ad015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 0x7f1ad015de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 -- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 msgr2=0x7f1ad0155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 0x7f1ad0155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 -- 172.21.15.22:0/1558189406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8009a70 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5888700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 0x7f1ad0155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad4886700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 0x7f1ad015de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad5087700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c4f0 tx=0x7f1acc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0052d0 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc005430 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00b5420 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003930 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00b59e0 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.185+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad004e4c0 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.189+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1acc003a90 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.189+0000 7f1ac5ffb700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 0x7f1ab40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.189+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1acc0839b0 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.189+0000 7f1ad4886700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 0x7f1ab40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.189+0000 7f1ad4886700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 0x7f1ab40687a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0158600 tx=0x7f1ac0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.193+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1acc050b90 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.357+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ad006a680 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.357+0000 7f1ac5ffb700 1 -- 172.21.15.22:0/1558189406 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1acc0170a0 con 0x7f1ad00cf3a0 2023-06-23T18:04:22.370 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:22.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 msgr2=0x7f1ab40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 0x7f1ab40687a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0158600 tx=0x7f1ac0009040 comp rx=0 tx=0).stop 2023-06-23T18:04:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 msgr2=0x7f1ad01560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c4f0 tx=0x7f1acc00c8b0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 shutdown_connections 2023-06-23T18:04:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ab40662f0 0x7f1ab40687a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ad00d1f30 0x7f1ad0155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ad00d0730 0x7f1ad015de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 --2- 172.21.15.22:0/1558189406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad00cf3a0 0x7f1ad01560e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 >> 172.21.15.22:0/1558189406 conn(0x7f1ad006a280 msgr2=0x7f1ad00b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 shutdown_connections 2023-06-23T18:04:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.361+0000 7f1ad6089700 1 -- 172.21.15.22:0/1558189406 wait complete. 2023-06-23T18:04:22.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:22.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 Processor -- start 2023-06-23T18:04:22.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 -- start start 2023-06-23T18:04:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3210036320 0x7f32100e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f32100514c0 0x7f3210039d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3210038980 0x7f3210038e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3210039360 con 0x7f32100514c0 2023-06-23T18:04:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100e5260 con 0x7f3210036320 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.493+0000 7f321caa3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210053500 con 0x7f3210038980 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f32100514c0 0x7f3210039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f32100514c0 0x7f3210039d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49234/0 (socket says 172.21.15.22:49234) 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 -- 172.21.15.22:0/283224632 learned_addr learned my addr 172.21.15.22:0/283224632 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3217fff700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3210038980 0x7f3210038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f32177fe700 1 --2- 172.21.15.22:0/283224632 >> v2:172.21.15.169:3300/0 conn(0x7f3210036320 0x7f32100e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 -- 172.21.15.22:0/283224632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3210038980 msgr2=0x7f3210038e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3210038980 0x7f3210038e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 -- 172.21.15.22:0/283224632 >> v2:172.21.15.169:3300/0 conn(0x7f3210036320 msgr2=0x7f32100e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 --2- 172.21.15.22:0/283224632 >> v2:172.21.15.169:3300/0 conn(0x7f3210036320 0x7f32100e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 -- 172.21.15.22:0/283224632 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210051950 con 0x7f32100514c0 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f3216ffd700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32100514c0 0x7f3210039d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f321000ae80 tx=0x7f3208009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7496e99e6d2d6327 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f32167fc700 1 -- 172.21.15.22:0/283224632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011830 con 0x7f32100514c0 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f32167fc700 1 -- 172.21.15.22:0/283224632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208011990 con 0x7f32100514c0 2023-06-23T18:04:22.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f32167fc700 1 -- 172.21.15.22:0/283224632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011d20 con 0x7f32100514c0 2023-06-23T18:04:22.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- 172.21.15.22:0/283224632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32100514c0 msgr2=0x7f3210039d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32100514c0 0x7f3210039d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f321000ae80 tx=0x7f3208009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- 172.21.15.22:0/283224632 shutdown_connections 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3210038980 0x7f3210038e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 --2- 172.21.15.22:0/283224632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32100514c0 0x7f3210039d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 --2- 172.21.15.22:0/283224632 >> v2:172.21.15.169:3300/0 conn(0x7f3210036320 0x7f32100e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- 172.21.15.22:0/283224632 >> 172.21.15.22:0/283224632 conn(0x7f3210057b90 msgr2=0x7f3210053980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- 172.21.15.22:0/283224632 shutdown_connections 2023-06-23T18:04:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- 172.21.15.22:0/283224632 wait complete. 2023-06-23T18:04:22.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 Processor -- start 2023-06-23T18:04:22.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.497+0000 7f321caa3700 1 -- start start 2023-06-23T18:04:22.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 0x7f3210178750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32100514c0 0x7f3210033760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100dfc40 con 0x7f3210036320 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210032d10 con 0x7f3210038980 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210178c90 con 0x7f32100514c0 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49242/0 (socket says 172.21.15.22:49242) 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 -- 172.21.15.22:0/1106988782 learned_addr learned my addr 172.21.15.22:0/1106988782 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f3216ffd700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 0x7f3210178750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f3217fff700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32100514c0 0x7f3210033760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 -- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32100514c0 msgr2=0x7f3210033760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32100514c0 0x7f3210033760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 -- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 msgr2=0x7f3210178750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 0x7f3210178750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 -- 172.21.15.22:0/1106988782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009570 con 0x7f3210036320 2023-06-23T18:04:22.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f3216ffd700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 0x7f3210178750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f32177fe700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d800 tx=0x7f320c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c014620 con 0x7f3210036320 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c014780 con 0x7f3210036320 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321004c740 con 0x7f3210036320 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c014b10 con 0x7f3210036320 2023-06-23T18:04:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.501+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3210032f60 con 0x7f3210036320 2023-06-23T18:04:22.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321004cc60 con 0x7f3210036320 2023-06-23T18:04:22.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f320c014c70 con 0x7f3210036320 2023-06-23T18:04:22.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f31fffff700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 0x7f32040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f320c0845e0 con 0x7f3210036320 2023-06-23T18:04:22.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f3216ffd700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 0x7f32040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.505+0000 7f3216ffd700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 0x7f32040687a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3208000c00 tx=0x7f3208008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.513+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f320c0516f0 con 0x7f3210036320 2023-06-23T18:04:22.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.673+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32100059c0 con 0x7f3210036320 2023-06-23T18:04:22.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.673+0000 7f31fffff700 1 -- 172.21.15.22:0/1106988782 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f320c054d10 con 0x7f3210036320 2023-06-23T18:04:22.686 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:22.687 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 msgr2=0x7f32040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 0x7f32040687a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3208000c00 tx=0x7f3208008030 comp rx=0 tx=0).stop 2023-06-23T18:04:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 msgr2=0x7f321004c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d800 tx=0x7f320c00dbc0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 shutdown_connections 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32100514c0 0x7f3210033760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f32040662f0 0x7f32040687a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3210038980 0x7f3210178750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 --2- 172.21.15.22:0/1106988782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3210036320 0x7f321004c080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 >> 172.21.15.22:0/1106988782 conn(0x7f3210057b90 msgr2=0x7f3210053910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 shutdown_connections 2023-06-23T18:04:22.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.677+0000 7f321caa3700 1 -- 172.21.15.22:0/1106988782 wait complete. 2023-06-23T18:04:22.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:22.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:22.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 Processor -- start 2023-06-23T18:04:22.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 -- start start 2023-06-23T18:04:22.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc190160480 0x7fc1900ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900ad070 0x7fc1900b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1900affc0 con 0x7fc1900c1040 2023-06-23T18:04:22.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1900ad950 con 0x7fc190160480 2023-06-23T18:04:22.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.817+0000 7fc18ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900cc3a0 con 0x7fc1900ad070 2023-06-23T18:04:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc190160480 0x7fc1900ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc190160480 0x7fc1900ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49256/0 (socket says 172.21.15.22:49256) 2023-06-23T18:04:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 -- 172.21.15.22:0/3534094797 learned_addr learned my addr 172.21.15.22:0/3534094797 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc1867fc700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900ad070 0x7fc1900b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 -- 172.21.15.22:0/3534094797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900ad070 msgr2=0x7fc1900b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900ad070 0x7fc1900b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 -- 172.21.15.22:0/3534094797 >> v1:172.21.15.22:6789/0 conn(0x7fc1900c1040 legacy=0x7fc190163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 -- 172.21.15.22:0/3534094797 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1900b0990 con 0x7fc190160480 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18e7fc700 1 -- 172.21.15.22:0/3534094797 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646695158 0 0) 0x7fc1900affc0 con 0x7fc1900c1040 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18effd700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc190160480 0x7fc1900ccf50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc180009940 tx=0x7fc180009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b13a768425af643 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18e7fc700 1 -- 172.21.15.22:0/3534094797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000ba20 con 0x7fc190160480 2023-06-23T18:04:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18e7fc700 1 -- 172.21.15.22:0/3534094797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc180009e90 con 0x7fc190160480 2023-06-23T18:04:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18e7fc700 1 -- 172.21.15.22:0/3534094797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000bda0 con 0x7fc190160480 2023-06-23T18:04:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 -- 172.21.15.22:0/3534094797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc190160480 msgr2=0x7fc1900ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc190160480 0x7fc1900ccf50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc180009940 tx=0x7fc180009c50 comp rx=0 tx=0).stop 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 -- 172.21.15.22:0/3534094797 shutdown_connections 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900ad070 0x7fc1900b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 --2- 172.21.15.22:0/3534094797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc190160480 0x7fc1900ccf50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 -- 172.21.15.22:0/3534094797 >> 172.21.15.22:0/3534094797 conn(0x7fc19006a5f0 msgr2=0x7fc1900caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 -- 172.21.15.22:0/3534094797 shutdown_connections 2023-06-23T18:04:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.821+0000 7fc18ffff700 1 -- 172.21.15.22:0/3534094797 wait complete. 2023-06-23T18:04:22.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 Processor -- start 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- start start 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900c1040 0x7fc1900c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc190160480 0x7fc1900aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c7f50 con 0x7fc1900ad070 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c6310 con 0x7fc190160480 2023-06-23T18:04:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c65d0 con 0x7fc1900c1040 2023-06-23T18:04:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49258/0 (socket says 172.21.15.22:49258) 2023-06-23T18:04:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 -- 172.21.15.22:0/1476849598 learned_addr learned my addr 172.21.15.22:0/1476849598 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc1867fc700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900c1040 0x7fc1900c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18f7fe700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc190160480 0x7fc1900aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 -- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900c1040 msgr2=0x7fc1900c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900c1040 0x7fc1900c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 -- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc190160480 msgr2=0x7fc1900aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc190160480 0x7fc1900aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 -- 172.21.15.22:0/1476849598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009570 con 0x7fc1900ad070 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18effd700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc180004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180011830 con 0x7fc1900ad070 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18001e070 con 0x7fc1900ad070 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1900c68b0 con 0x7fc1900ad070 2023-06-23T18:04:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180011bd0 con 0x7fc1900ad070 2023-06-23T18:04:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.825+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1900c6e70 con 0x7fc1900ad070 2023-06-23T18:04:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19004e4c0 con 0x7fc1900ad070 2023-06-23T18:04:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc180011d30 con 0x7fc1900ad070 2023-06-23T18:04:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc187fff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 0x7fc1700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc180020070 con 0x7fc1900ad070 2023-06-23T18:04:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc1867fc700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 0x7fc1700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.829+0000 7fc1867fc700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 0x7fc1700687a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1900cec20 tx=0x7fc178008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:22.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.837+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc1800525b0 con 0x7fc1900ad070 2023-06-23T18:04:23.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.997+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc190071920 con 0x7fc1900ad070 2023-06-23T18:04:23.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:22.997+0000 7fc187fff700 1 -- 172.21.15.22:0/1476849598 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc180055bd0 con 0x7fc1900ad070 2023-06-23T18:04:23.010 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 msgr2=0x7fc1700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 0x7fc1700687a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1900cec20 tx=0x7fc178008040 comp rx=0 tx=0).stop 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 msgr2=0x7fc19015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc180004770 comp rx=0 tx=0).stop 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 shutdown_connections 2023-06-23T18:04:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc1700662f0 0x7fc1700687a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc190160480 0x7fc1900aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900c1040 0x7fc1900c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 --2- 172.21.15.22:0/1476849598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1900ad070 0x7fc19015d810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 >> 172.21.15.22:0/1476849598 conn(0x7fc19006a5f0 msgr2=0x7fc1900c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 shutdown_connections 2023-06-23T18:04:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.001+0000 7fc18ffff700 1 -- 172.21.15.22:0/1476849598 wait complete. 2023-06-23T18:04:23.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.133+0000 7f2f5902e700 1 Processor -- start 2023-06-23T18:04:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.133+0000 7f2f5902e700 1 -- start start 2023-06-23T18:04:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f5415bf10 0x7f2f5406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f5406c510 0x7f2f540c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5406f660 0x7f2f540c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5406cf70 con 0x7f2f5415bf10 2023-06-23T18:04:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5406c9b0 con 0x7f2f5406c510 2023-06-23T18:04:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5902e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5406cc90 con 0x7f2f5406f660 2023-06-23T18:04:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f5415bf10 0x7f2f5406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2f5415bf10 0x7f2f5406bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49264/0 (socket says 172.21.15.22:49264) 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 -- 172.21.15.22:0/1301708829 learned_addr learned my addr 172.21.15.22:0/1301708829 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f4bfff700 1 --2- 172.21.15.22:0/1301708829 >> v2:172.21.15.169:3300/0 conn(0x7f2f5406c510 0x7f2f540c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f5882d700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5406f660 0x7f2f540c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 -- 172.21.15.22:0/1301708829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5406f660 msgr2=0x7f2f540c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:23.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5406f660 0x7f2f540c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 -- 172.21.15.22:0/1301708829 >> v2:172.21.15.169:3300/0 conn(0x7f2f5406c510 msgr2=0x7f2f540c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 --2- 172.21.15.22:0/1301708829 >> v2:172.21.15.169:3300/0 conn(0x7f2f5406c510 0x7f2f540c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 -- 172.21.15.22:0/1301708829 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540cc5e0 con 0x7f2f5415bf10 2023-06-23T18:04:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f53fff700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5415bf10 0x7f2f5406bfd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f5406e450 tx=0x7f2f44009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e284dbe776fa736 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f537fe700 1 -- 172.21.15.22:0/1301708829 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011830 con 0x7f2f5415bf10 2023-06-23T18:04:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f537fe700 1 -- 172.21.15.22:0/1301708829 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44011990 con 0x7f2f5415bf10 2023-06-23T18:04:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.137+0000 7f2f537fe700 1 -- 172.21.15.22:0/1301708829 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011d20 con 0x7f2f5415bf10 2023-06-23T18:04:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- 172.21.15.22:0/1301708829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5415bf10 msgr2=0x7f2f5406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5415bf10 0x7f2f5406bfd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f5406e450 tx=0x7f2f44009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- 172.21.15.22:0/1301708829 shutdown_connections 2023-06-23T18:04:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5406f660 0x7f2f540c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- 172.21.15.22:0/1301708829 >> v2:172.21.15.169:3300/0 conn(0x7f2f5406c510 0x7f2f540c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- 172.21.15.22:0/1301708829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5415bf10 0x7f2f5406bfd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- 172.21.15.22:0/1301708829 >> 172.21.15.22:0/1301708829 conn(0x7f2f5406a5f0 msgr2=0x7f2f540caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- 172.21.15.22:0/1301708829 shutdown_connections 2023-06-23T18:04:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- 172.21.15.22:0/1301708829 wait complete. 2023-06-23T18:04:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 Processor -- start 2023-06-23T18:04:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 -- start start 2023-06-23T18:04:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5406f660 0x7f2f540c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.141+0000 7f2f5902e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5415bf10 0x7f2f540c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b36a0 con 0x7f2f5406c510 2023-06-23T18:04:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540c2720 con 0x7f2f5406f660 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540c2af0 con 0x7f2f5415bf10 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49270/0 (socket says 172.21.15.22:49270) 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 -- 172.21.15.22:0/3927760194 learned_addr learned my addr 172.21.15.22:0/3927760194 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f4bfff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5406f660 0x7f2f540c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5882d700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5415bf10 0x7f2f540c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 -- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5415bf10 msgr2=0x7f2f540c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5415bf10 0x7f2f540c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 -- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5406f660 msgr2=0x7f2f540c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5406f660 0x7f2f540c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 -- 172.21.15.22:0/3927760194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009570 con 0x7f2f5406c510 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f53fff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009f40 tx=0x7f2f44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004240 con 0x7f2f5406c510 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4401f070 con 0x7f2f5406c510 2023-06-23T18:04:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540c7590 con 0x7f2f5406c510 2023-06-23T18:04:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004600 con 0x7f2f5406c510 2023-06-23T18:04:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540c7b30 con 0x7f2f5406c510 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.145+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5404e4c0 con 0x7f2f5406c510 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.149+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2f44021030 con 0x7f2f5406c510 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.149+0000 7f2f517fa700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 0x7f2f380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.149+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f44084f70 con 0x7f2f5406c510 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.149+0000 7f2f4bfff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 0x7f2f380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.149+0000 7f2f4bfff700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 0x7f2f380687a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c1950 tx=0x7f2f3c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.157+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2f440b5700 con 0x7f2f5406c510 2023-06-23T18:04:23.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2f540acc70 con 0x7f2f5406c510 2023-06-23T18:04:23.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f517fa700 1 -- 172.21.15.22:0/3927760194 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2f44015020 con 0x7f2f5406c510 2023-06-23T18:04:23.337 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 msgr2=0x7f2f380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 0x7f2f380687a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c1950 tx=0x7f2f3c009240 comp rx=0 tx=0).stop 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 msgr2=0x7f2f540c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.325+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009f40 tx=0x7f2f44004770 comp rx=0 tx=0).stop 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 shutdown_connections 2023-06-23T18:04:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f5415bf10 0x7f2f540c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2f380662f0 0x7f2f380687a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5406f660 0x7f2f540c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 --2- 172.21.15.22:0/3927760194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5406c510 0x7f2f540c21e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 >> 172.21.15.22:0/3927760194 conn(0x7f2f5406a5f0 msgr2=0x7f2f540acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 shutdown_connections 2023-06-23T18:04:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.329+0000 7f2f5902e700 1 -- 172.21.15.22:0/3927760194 wait complete. 2023-06-23T18:04:23.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.457+0000 7fde668eb700 1 Processor -- start 2023-06-23T18:04:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.457+0000 7fde668eb700 1 -- start start 2023-06-23T18:04:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde668eb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fde600b0200 0x7fde600cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde668eb700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde600ca6c0 con 0x7fde60070e00 2023-06-23T18:04:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde668eb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde600cbde0 con 0x7fde60070a30 2023-06-23T18:04:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde668eb700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600ca9a0 con 0x7fde600b0200 2023-06-23T18:04:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde650e8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fde600b0200 0x7fde600cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde660ea700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fde60070e00 0x7fde600c9ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33980/0 (socket says 172.21.15.22:33980) 2023-06-23T18:04:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde660ea700 1 -- 172.21.15.22:0/1723582469 learned_addr learned my addr 172.21.15.22:0/1723582469 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde648e7700 1 -- 172.21.15.22:0/1723582469 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200357162 0 0) 0x7fde600ca6c0 con 0x7fde60070e00 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde648e7700 1 -- 172.21.15.22:0/1723582469 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde4c0035d0 con 0x7fde60070e00 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde650e8700 1 -- 172.21.15.22:0/1723582469 >> v1:172.21.15.169:6789/0 conn(0x7fde60070a30 legacy=0x7fde600afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde650e8700 1 -- 172.21.15.22:0/1723582469 >> v1:172.21.15.22:6789/0 conn(0x7fde60070e00 legacy=0x7fde600c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde650e8700 1 -- 172.21.15.22:0/1723582469 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600cc0c0 con 0x7fde600b0200 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde650e8700 1 --2- 172.21.15.22:0/1723582469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde600b0200 0x7fde600cc8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009850 tx=0x7fde5c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2d790eea1809e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde648e7700 1 -- 172.21.15.22:0/1723582469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c011830 con 0x7fde600b0200 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde648e7700 1 -- 172.21.15.22:0/1723582469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c011990 con 0x7fde600b0200 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.461+0000 7fde648e7700 1 -- 172.21.15.22:0/1723582469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c011d40 con 0x7fde600b0200 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- 172.21.15.22:0/1723582469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde600b0200 msgr2=0x7fde600cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 --2- 172.21.15.22:0/1723582469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde600b0200 0x7fde600cc8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009850 tx=0x7fde5c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- 172.21.15.22:0/1723582469 shutdown_connections 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 --2- 172.21.15.22:0/1723582469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde600b0200 0x7fde600cc8c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- 172.21.15.22:0/1723582469 >> 172.21.15.22:0/1723582469 conn(0x7fde6006a5f0 msgr2=0x7fde600caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- 172.21.15.22:0/1723582469 shutdown_connections 2023-06-23T18:04:23.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- 172.21.15.22:0/1723582469 wait complete. 2023-06-23T18:04:23.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 Processor -- start 2023-06-23T18:04:23.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.465+0000 7fde668eb700 1 -- start start 2023-06-23T18:04:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde60070e00 0x7fde600cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde600b0200 0x7fde600cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c24c0 con 0x7fde60070a30 2023-06-23T18:04:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c1b90 con 0x7fde60070e00 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c1e00 con 0x7fde600b0200 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49298/0 (socket says 172.21.15.22:49298) 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 -- 172.21.15.22:0/38671822 learned_addr learned my addr 172.21.15.22:0/38671822 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde650e8700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde60070e00 0x7fde600cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde660ea700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde600b0200 0x7fde600cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 -- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde600b0200 msgr2=0x7fde600cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde600b0200 0x7fde600cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 -- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde60070e00 msgr2=0x7fde600cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde60070e00 0x7fde600cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 -- 172.21.15.22:0/38671822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0094b0 con 0x7fde60070a30 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde658e9700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c530 tx=0x7fde5400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5400cea0 con 0x7fde60070a30 2023-06-23T18:04:23.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540052d0 con 0x7fde60070a30 2023-06-23T18:04:23.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde600c20e0 con 0x7fde60070a30 2023-06-23T18:04:23.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540036a0 con 0x7fde60070a30 2023-06-23T18:04:23.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde600b19a0 con 0x7fde60070a30 2023-06-23T18:04:23.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.469+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6004e4c0 con 0x7fde60070a30 2023-06-23T18:04:23.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.473+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fde54003800 con 0x7fde60070a30 2023-06-23T18:04:23.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.473+0000 7fde527fc700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 0x7fde440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.473+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fde540837d0 con 0x7fde60070a30 2023-06-23T18:04:23.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.473+0000 7fde650e8700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 0x7fde440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.473+0000 7fde650e8700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 0x7fde440687a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde600b05f0 tx=0x7fde5c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.481+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fde54050940 con 0x7fde60070a30 2023-06-23T18:04:23.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.641+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde600b3ff0 con 0x7fde60070a30 2023-06-23T18:04:23.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.641+0000 7fde527fc700 1 -- 172.21.15.22:0/38671822 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde54053f60 con 0x7fde60070a30 2023-06-23T18:04:23.653 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:23.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 msgr2=0x7fde440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 0x7fde440687a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde600b05f0 tx=0x7fde5c008030 comp rx=0 tx=0).stop 2023-06-23T18:04:23.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 msgr2=0x7fde600b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c530 tx=0x7fde5400c840 comp rx=0 tx=0).stop 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 shutdown_connections 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde600b0200 0x7fde600cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fde440662f0 0x7fde440687a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde60070e00 0x7fde600cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 --2- 172.21.15.22:0/38671822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde60070a30 0x7fde600b1460 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 >> 172.21.15.22:0/38671822 conn(0x7fde6006a5f0 msgr2=0x7fde6006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 shutdown_connections 2023-06-23T18:04:23.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.645+0000 7fde668eb700 1 -- 172.21.15.22:0/38671822 wait complete. 2023-06-23T18:04:23.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.777+0000 7f8b1359e700 1 Processor -- start 2023-06-23T18:04:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.777+0000 7f8b1359e700 1 -- start start 2023-06-23T18:04:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b140cc430 0x7f8b140b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b140cd870 0x7f8b1406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b1406c150 0x7f8b1406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b140cc800 con 0x7f8b140cd870 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b140b2c60 con 0x7f8b140cc430 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1406cb10 con 0x7f8b1406c150 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b140cd870 0x7f8b1406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b140cd870 0x7f8b1406f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49314/0 (socket says 172.21.15.22:49314) 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 -- 172.21.15.22:0/1217793026 learned_addr learned my addr 172.21.15.22:0/1217793026 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b12d9d700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b1406c150 0x7f8b1406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1259c700 1 --2- 172.21.15.22:0/1217793026 >> v2:172.21.15.169:3300/0 conn(0x7f8b140cc430 0x7f8b140b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 -- 172.21.15.22:0/1217793026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b1406c150 msgr2=0x7f8b1406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b1406c150 0x7f8b1406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 -- 172.21.15.22:0/1217793026 >> v2:172.21.15.169:3300/0 conn(0x7f8b140cc430 msgr2=0x7f8b140b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/1217793026 >> v2:172.21.15.169:3300/0 conn(0x7f8b140cc430 0x7f8b140b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 -- 172.21.15.22:0/1217793026 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1415fa00 con 0x7f8b140cd870 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b140cd870 0x7f8b1406f2e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b14071020 tx=0x7f8afc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd7a08e97b5acc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b11d9b700 1 -- 172.21.15.22:0/1217793026 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc011830 con 0x7f8b140cd870 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b11d9b700 1 -- 172.21.15.22:0/1217793026 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc011990 con 0x7f8b140cd870 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b11d9b700 1 -- 172.21.15.22:0/1217793026 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc011d20 con 0x7f8b140cd870 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- 172.21.15.22:0/1217793026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b140cd870 msgr2=0x7f8b1406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b140cd870 0x7f8b1406f2e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b14071020 tx=0x7f8afc009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- 172.21.15.22:0/1217793026 shutdown_connections 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b1406c150 0x7f8b1406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- 172.21.15.22:0/1217793026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b140cd870 0x7f8b1406f2e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 --2- 172.21.15.22:0/1217793026 >> v2:172.21.15.169:3300/0 conn(0x7f8b140cc430 0x7f8b140b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- 172.21.15.22:0/1217793026 >> 172.21.15.22:0/1217793026 conn(0x7f8b140ccd60 msgr2=0x7f8b140cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.781+0000 7f8b1359e700 1 -- 172.21.15.22:0/1217793026 shutdown_connections 2023-06-23T18:04:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.785+0000 7f8b1359e700 1 -- 172.21.15.22:0/1217793026 wait complete. 2023-06-23T18:04:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.785+0000 7f8b1359e700 1 Processor -- start 2023-06-23T18:04:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.785+0000 7f8b1359e700 1 -- start start 2023-06-23T18:04:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 0x7f8b1415db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 0x7f8b141e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140c8c90 con 0x7f8b1406c150 2023-06-23T18:04:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140c7660 con 0x7f8b140cd870 2023-06-23T18:04:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1415e2c0 con 0x7f8b140cc430 2023-06-23T18:04:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49318/0 (socket says 172.21.15.22:49318) 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 -- 172.21.15.22:0/3244372159 learned_addr learned my addr 172.21.15.22:0/3244372159 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 0x7f8b1415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 -- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 msgr2=0x7f8b1415db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 0x7f8b1415db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b12d9d700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 0x7f8b141e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 -- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 msgr2=0x7f8b141e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 0x7f8b141e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 -- 172.21.15.22:0/3244372159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009570 con 0x7f8b1406c150 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 0x7f8b1415db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b12d9d700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 0x7f8b141e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1259c700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400e9d0 tx=0x7f8b0400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04009720 con 0x7f8b1406c150 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0400c850 con 0x7f8b1406c150 2023-06-23T18:04:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1415e5a0 con 0x7f8b1406c150 2023-06-23T18:04:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400cbc0 con 0x7f8b1406c150 2023-06-23T18:04:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1415ee40 con 0x7f8b1406c150 2023-06-23T18:04:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.789+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1415e9a0 con 0x7f8b1406c150 2023-06-23T18:04:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.793+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8b0400cd20 con 0x7f8b1406c150 2023-06-23T18:04:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.793+0000 7f8b0b7fe700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 0x7f8af80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.793+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 0x7f8af80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.793+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8b040841f0 con 0x7f8b1406c150 2023-06-23T18:04:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.793+0000 7f8b09d9b700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 0x7f8af80687a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b141e78c0 tx=0x7f8afc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:23.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.801+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8b04054d80 con 0x7f8b1406c150 2023-06-23T18:04:23.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.969+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b1404e4c0 con 0x7f8b1406c150 2023-06-23T18:04:23.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.969+0000 7f8b0b7fe700 1 -- 172.21.15.22:0/3244372159 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b04054910 con 0x7f8b1406c150 2023-06-23T18:04:23.981 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:23.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 msgr2=0x7f8af80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 0x7f8af80687a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b141e78c0 tx=0x7f8afc008030 comp rx=0 tx=0).stop 2023-06-23T18:04:23.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 msgr2=0x7f8b140ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:23.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400e9d0 tx=0x7f8b0400ed90 comp rx=0 tx=0).stop 2023-06-23T18:04:23.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 shutdown_connections 2023-06-23T18:04:23.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8af80662f0 0x7f8af80687a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b140cd870 0x7f8b141e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b140cc430 0x7f8b1415db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 --2- 172.21.15.22:0/3244372159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b1406c150 0x7f8b140ca050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:23.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 >> 172.21.15.22:0/3244372159 conn(0x7f8b140ccd60 msgr2=0x7f8b140b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:23.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 shutdown_connections 2023-06-23T18:04:23.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:23.973+0000 7f8b1359e700 1 -- 172.21.15.22:0/3244372159 wait complete. 2023-06-23T18:04:24.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:24.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 Processor -- start 2023-06-23T18:04:24.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 -- start start 2023-06-23T18:04:24.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 0x7f9d2806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d2806c510 0x7f9d280c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d280cddf0 con 0x7f9d2806f660 2023-06-23T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2806cf70 con 0x7f9d2806c510 2023-06-23T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d2d853700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2806c9b0 con 0x7f9d2815bf10 2023-06-23T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d27fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d2806c510 0x7f9d280c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d27fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d2806c510 0x7f9d280c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49324/0 (socket says 172.21.15.22:49324) 2023-06-23T18:04:24.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.113+0000 7f9d27fff700 1 -- 172.21.15.22:0/2340074319 learned_addr learned my addr 172.21.15.22:0/2340074319 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2c851700 1 --2- 172.21.15.22:0/2340074319 >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 0x7f9d2806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d27fff700 1 -- 172.21.15.22:0/2340074319 >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 msgr2=0x7f9d2806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:24.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d27fff700 1 --2- 172.21.15.22:0/2340074319 >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 0x7f9d2806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d27fff700 1 -- 172.21.15.22:0/2340074319 >> v1:172.21.15.169:6789/0 conn(0x7f9d2806f660 legacy=0x7f9d2815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d27fff700 1 -- 172.21.15.22:0/2340074319 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2806cc90 con 0x7f9d2806c510 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2c851700 1 --2- 172.21.15.22:0/2340074319 >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 0x7f9d2806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d27fff700 1 --2- 172.21.15.22:0/2340074319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806c510 0x7f9d280c89f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d2806dcd0 tx=0x7f9d14009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7b0bfa28b36639 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d277fe700 1 -- 172.21.15.22:0/2340074319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400b8a0 con 0x7f9d2806c510 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d277fe700 1 -- 172.21.15.22:0/2340074319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400ba00 con 0x7f9d2806c510 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d277fe700 1 -- 172.21.15.22:0/2340074319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400bd90 con 0x7f9d2806c510 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- 172.21.15.22:0/2340074319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806c510 msgr2=0x7f9d280c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2340074319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806c510 0x7f9d280c89f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d2806dcd0 tx=0x7f9d14009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- 172.21.15.22:0/2340074319 shutdown_connections 2023-06-23T18:04:24.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2340074319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806c510 0x7f9d280c89f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2340074319 >> v2:172.21.15.169:3300/0 conn(0x7f9d2815bf10 0x7f9d2806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- 172.21.15.22:0/2340074319 >> 172.21.15.22:0/2340074319 conn(0x7f9d2806a5f0 msgr2=0x7f9d280caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- 172.21.15.22:0/2340074319 shutdown_connections 2023-06-23T18:04:24.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- 172.21.15.22:0/2340074319 wait complete. 2023-06-23T18:04:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 Processor -- start 2023-06-23T18:04:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.117+0000 7f9d2d853700 1 -- start start 2023-06-23T18:04:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2806c510 0x7f9d280b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2815bf10 0x7f9d281518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b3e40 con 0x7f9d2806f660 2023-06-23T18:04:24.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b2da0 con 0x7f9d2815bf10 2023-06-23T18:04:24.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b3080 con 0x7f9d2806c510 2023-06-23T18:04:24.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49334/0 (socket says 172.21.15.22:49334) 2023-06-23T18:04:24.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 -- 172.21.15.22:0/2311099983 learned_addr learned my addr 172.21.15.22:0/2311099983 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d052700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2815bf10 0x7f9d281518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2c851700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2806c510 0x7f9d280b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 -- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2806c510 msgr2=0x7f9d280b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2806c510 0x7f9d280b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 -- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2815bf10 msgr2=0x7f9d281518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2815bf10 0x7f9d281518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 -- 172.21.15.22:0/2311099983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009570 con 0x7f9d2806f660 2023-06-23T18:04:24.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d27fff700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d14000c00 tx=0x7f9d14004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d140038c0 con 0x7f9d2806f660 2023-06-23T18:04:24.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1401e070 con 0x7f9d2806f660 2023-06-23T18:04:24.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d280b3360 con 0x7f9d2806f660 2023-06-23T18:04:24.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14003c80 con 0x7f9d2806f660 2023-06-23T18:04:24.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.121+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d280b3920 con 0x7f9d2806f660 2023-06-23T18:04:24.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2804e4c0 con 0x7f9d2806f660 2023-06-23T18:04:24.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9d14011420 con 0x7f9d2806f660 2023-06-23T18:04:24.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d257fa700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 0x7f9d0c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9d14020070 con 0x7f9d2806f660 2023-06-23T18:04:24.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d2c851700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 0x7f9d0c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.125+0000 7f9d2c851700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 0x7f9d0c0687a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0099f0 tx=0x7f9d1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.133+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f9d140527b0 con 0x7f9d2806f660 2023-06-23T18:04:24.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.297+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9d280cc5b0 con 0x7f9d2806f660 2023-06-23T18:04:24.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d257fa700 1 -- 172.21.15.22:0/2311099983 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d14055dd0 con 0x7f9d2806f660 2023-06-23T18:04:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 msgr2=0x7f9d0c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 0x7f9d0c0687a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0099f0 tx=0x7f9d1c008040 comp rx=0 tx=0).stop 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 msgr2=0x7f9d280b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d14000c00 tx=0x7f9d14004770 comp rx=0 tx=0).stop 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 shutdown_connections 2023-06-23T18:04:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9d0c0662f0 0x7f9d0c0687a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2815bf10 0x7f9d281518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d2806f660 0x7f9d280b0ea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 --2- 172.21.15.22:0/2311099983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2806c510 0x7f9d280b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 >> 172.21.15.22:0/2311099983 conn(0x7f9d2806a5f0 msgr2=0x7f9d280c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 shutdown_connections 2023-06-23T18:04:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.301+0000 7f9d2d853700 1 -- 172.21.15.22:0/2311099983 wait complete. 2023-06-23T18:04:24.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:24.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 Processor -- start 2023-06-23T18:04:24.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 -- start start 2023-06-23T18:04:24.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97b015c280 0x7f97b00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97b006d110 0x7f97b015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00ca370 con 0x7f97b015beb0 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00c8790 con 0x7f97b015c280 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b50d7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b0160780 con 0x7f97b006d110 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97b015c280 0x7f97b00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97b015c280 0x7f97b00c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49336/0 (socket says 172.21.15.22:49336) 2023-06-23T18:04:24.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97affff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f97b015beb0 0x7f97b0070ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:33986/0 (socket says 172.21.15.22:33986) 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 -- 172.21.15.22:0/1951515510 learned_addr learned my addr 172.21.15.22:0/1951515510 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97b48d6700 1 --2- 172.21.15.22:0/1951515510 >> v2:172.21.15.169:3300/0 conn(0x7f97b006d110 0x7f97b015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 -- 172.21.15.22:0/1951515510 >> v2:172.21.15.169:3300/0 conn(0x7f97b006d110 msgr2=0x7f97b015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 --2- 172.21.15.22:0/1951515510 >> v2:172.21.15.169:3300/0 conn(0x7f97b006d110 0x7f97b015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 -- 172.21.15.22:0/1951515510 >> v1:172.21.15.22:6789/0 conn(0x7f97b015beb0 legacy=0x7f97b0070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 -- 172.21.15.22:0/1951515510 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0071920 con 0x7f97b015c280 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97af7fe700 1 --2- 172.21.15.22:0/1951515510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00c8f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97b0071b10 tx=0x7f9798009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e2d8ad5495dc0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97aeffd700 1 -- 172.21.15.22:0/1951515510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798004230 con 0x7f97b015c280 2023-06-23T18:04:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97aeffd700 1 -- 172.21.15.22:0/1951515510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9798004390 con 0x7f97b015c280 2023-06-23T18:04:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.429+0000 7f97aeffd700 1 -- 172.21.15.22:0/1951515510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980036c0 con 0x7f97b015c280 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- 172.21.15.22:0/1951515510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 msgr2=0x7f97b00c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- 172.21.15.22:0/1951515510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00c8f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97b0071b10 tx=0x7f9798009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- 172.21.15.22:0/1951515510 shutdown_connections 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- 172.21.15.22:0/1951515510 >> v2:172.21.15.169:3300/0 conn(0x7f97b006d110 0x7f97b015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- 172.21.15.22:0/1951515510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00c8f50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- 172.21.15.22:0/1951515510 >> 172.21.15.22:0/1951515510 conn(0x7f97b006a5f0 msgr2=0x7f97b00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- 172.21.15.22:0/1951515510 shutdown_connections 2023-06-23T18:04:24.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- 172.21.15.22:0/1951515510 wait complete. 2023-06-23T18:04:24.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 Processor -- start 2023-06-23T18:04:24.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- start start 2023-06-23T18:04:24.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97b006d110 0x7f97b0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b015beb0 0x7f97b00991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00af9c0 con 0x7f97b015c280 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0156d80 con 0x7f97b015beb0 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b50d7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00aebf0 con 0x7f97b006d110 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b48d6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b48d6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49342/0 (socket says 172.21.15.22:49342) 2023-06-23T18:04:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97b48d6700 1 -- 172.21.15.22:0/265600956 learned_addr learned my addr 172.21.15.22:0/265600956 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97af7fe700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b015beb0 0x7f97b00991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.433+0000 7f97affff700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97b006d110 0x7f97b0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 -- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97b006d110 msgr2=0x7f97b0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97b006d110 0x7f97b0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 -- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b015beb0 msgr2=0x7f97b00991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b015beb0 0x7f97b00991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 -- 172.21.15.22:0/265600956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798009570 con 0x7f97b015c280 2023-06-23T18:04:24.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b48d6700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f97a8007e90 tx=0x7f97a800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800ed30 con 0x7f97b015c280 2023-06-23T18:04:24.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800ee90 con 0x7f97b015c280 2023-06-23T18:04:24.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8014660 con 0x7f97b015c280 2023-06-23T18:04:24.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b0099710 con 0x7f97b015c280 2023-06-23T18:04:24.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b00af120 con 0x7f97b015c280 2023-06-23T18:04:24.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b004e4c0 con 0x7f97b015c280 2023-06-23T18:04:24.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.437+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f97a80147c0 con 0x7f97b015c280 2023-06-23T18:04:24.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.441+0000 7f97acff9700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 0x7f97900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.441+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97a8084610 con 0x7f97b015c280 2023-06-23T18:04:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.441+0000 7f97affff700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 0x7f97900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.441+0000 7f97affff700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 0x7f97900687a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000f60 tx=0x7f97a0007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.445+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f97a80517a0 con 0x7f97b015c280 2023-06-23T18:04:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.609+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b00c9b70 con 0x7f97b015c280 2023-06-23T18:04:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.613+0000 7f97acff9700 1 -- 172.21.15.22:0/265600956 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f97a8054dc0 con 0x7f97b015c280 2023-06-23T18:04:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:24.626 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:03:42.145065+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:24.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 msgr2=0x7f97900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 0x7f97900687a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000f60 tx=0x7f97a0007f30 comp rx=0 tx=0).stop 2023-06-23T18:04:24.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 msgr2=0x7f97b00cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f97a8007e90 tx=0x7f97a800e3c0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 shutdown_connections 2023-06-23T18:04:24.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b015c280 0x7f97b00cbbc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f97900662f0 0x7f97900687a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b015beb0 0x7f97b00991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 --2- 172.21.15.22:0/265600956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97b006d110 0x7f97b0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 >> 172.21.15.22:0/265600956 conn(0x7f97b006a5f0 msgr2=0x7f97b00c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 shutdown_connections 2023-06-23T18:04:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.617+0000 7f97b50d7700 1 -- 172.21.15.22:0/265600956 wait complete. 2023-06-23T18:04:24.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:24.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:24.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 Processor -- start 2023-06-23T18:04:24.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 -- start start 2023-06-23T18:04:24.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f234806f660 0x7f23480c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 0x7f23480c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234806c6e0 con 0x7f234815bf10 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234806cf70 con 0x7f234806f660 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234f305700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cddf0 con 0x7f234815c2e0 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.753+0000 7f234db02700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f234806f660 0x7f23480c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234eb04700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 0x7f23480c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f234806f660 0x7f23480c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46676/0 (socket says 172.21.15.22:46676) 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234e303700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f234815bf10 0x7f234806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:45870/0 (socket says 172.21.15.22:45870) 2023-06-23T18:04:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 -- 172.21.15.22:0/3702499411 learned_addr learned my addr 172.21.15.22:0/3702499411 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 -- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 msgr2=0x7f23480c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 0x7f23480c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 -- 172.21.15.22:0/3702499411 >> v1:172.21.15.169:6789/0 conn(0x7f234815bf10 legacy=0x7f234806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 -- 172.21.15.22:0/3702499411 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234806c9b0 con 0x7f234806f660 2023-06-23T18:04:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234eb04700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 0x7f23480c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234db02700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234806f660 0x7f23480c89f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2338009910 tx=0x7f2338009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b33fdaf37d82a9f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234d301700 1 -- 172.21.15.22:0/3702499411 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338011830 con 0x7f234806f660 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234d301700 1 -- 172.21.15.22:0/3702499411 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338011990 con 0x7f234806f660 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234d301700 1 -- 172.21.15.22:0/3702499411 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338011d40 con 0x7f234806f660 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 -- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234806f660 msgr2=0x7f23480c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234806f660 0x7f23480c89f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2338009910 tx=0x7f2338009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 -- 172.21.15.22:0/3702499411 shutdown_connections 2023-06-23T18:04:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815c2e0 0x7f23480c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 --2- 172.21.15.22:0/3702499411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234806f660 0x7f23480c89f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 -- 172.21.15.22:0/3702499411 >> 172.21.15.22:0/3702499411 conn(0x7f234806a5f0 msgr2=0x7f23480caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 -- 172.21.15.22:0/3702499411 shutdown_connections 2023-06-23T18:04:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.757+0000 7f234f305700 1 -- 172.21.15.22:0/3702499411 wait complete. 2023-06-23T18:04:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 Processor -- start 2023-06-23T18:04:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- start start 2023-06-23T18:04:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 0x7f23480b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815bf10 0x7f23481f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480b11c0 con 0x7f234806f660 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480b1cf0 con 0x7f234815c2e0 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480c6860 con 0x7f234815bf10 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234e303700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 0x7f23480b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234db02700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815bf10 0x7f23481f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46692/0 (socket says 172.21.15.22:46692) 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234e303700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 0x7f23480b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49348/0 (socket says 172.21.15.22:49348) 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 -- 172.21.15.22:0/2905877085 learned_addr learned my addr 172.21.15.22:0/2905877085 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 -- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815bf10 msgr2=0x7f23481f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815bf10 0x7f23481f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 -- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 msgr2=0x7f23480b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 0x7f23480b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 -- 172.21.15.22:0/2905877085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009a70 con 0x7f234815c2e0 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234eb04700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f234000efc0 tx=0x7f234000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000ca90 con 0x7f234815c2e0 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000f040 con 0x7f234815c2e0 2023-06-23T18:04:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2338009570 con 0x7f234815c2e0 2023-06-23T18:04:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000ce10 con 0x7f234815c2e0 2023-06-23T18:04:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.761+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480c6210 con 0x7f234815c2e0 2023-06-23T18:04:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.765+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23480c69d0 con 0x7f234815c2e0 2023-06-23T18:04:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.765+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2340003a40 con 0x7f234815c2e0 2023-06-23T18:04:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.765+0000 7f2346ffd700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 0x7f23340687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.765+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2340084f90 con 0x7f234815c2e0 2023-06-23T18:04:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.765+0000 7f234e303700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 0x7f23340687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.769+0000 7f234e303700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 0x7f23340687a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f233c000f60 tx=0x7f233c007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.773+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2340055d20 con 0x7f234815c2e0 2023-06-23T18:04:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.937+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f234804e4c0 con 0x7f234815c2e0 2023-06-23T18:04:24.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.937+0000 7f2346ffd700 1 -- 172.21.15.22:0/2905877085 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23400180a0 con 0x7f234815c2e0 2023-06-23T18:04:24.950 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 msgr2=0x7f23340687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 0x7f23340687a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f233c000f60 tx=0x7f233c007ef0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 msgr2=0x7f23480cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f234000efc0 tx=0x7f234000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 shutdown_connections 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23340662f0 0x7f23340687a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f234815c2e0 0x7f23480cdff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f234815bf10 0x7f23481f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 --2- 172.21.15.22:0/2905877085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f234806f660 0x7f23480b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.941+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 >> 172.21.15.22:0/2905877085 conn(0x7f234806a5f0 msgr2=0x7f2348159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:24.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.945+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 shutdown_connections 2023-06-23T18:04:24.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:24.945+0000 7f234f305700 1 -- 172.21.15.22:0/2905877085 wait complete. 2023-06-23T18:04:24.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:25.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.077+0000 7f2cca15e700 1 Processor -- start 2023-06-23T18:04:25.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.077+0000 7f2cca15e700 1 -- start start 2023-06-23T18:04:25.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cc40d1f30 0x7f2cc40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2cc40d2360 0x7f2cc40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc40ccb30 con 0x7f2cc40cf3a0 2023-06-23T18:04:25.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc40d2880 con 0x7f2cc40d1f30 2023-06-23T18:04:25.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc40d2b60 con 0x7f2cc40d2360 2023-06-23T18:04:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cc40d1f30 0x7f2cc40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc915c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2cc40cf3a0 0x7f2cc40cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34000/0 (socket says 172.21.15.22:34000) 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cc40d1f30 0x7f2cc40d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49356/0 (socket says 172.21.15.22:49356) 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc915c700 1 -- 172.21.15.22:0/1250895857 learned_addr learned my addr 172.21.15.22:0/1250895857 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc995d700 1 --2- 172.21.15.22:0/1250895857 >> v2:172.21.15.169:3300/0 conn(0x7f2cc40d2360 0x7f2cc40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 -- 172.21.15.22:0/1250895857 >> v2:172.21.15.169:3300/0 conn(0x7f2cc40d2360 msgr2=0x7f2cc40d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 --2- 172.21.15.22:0/1250895857 >> v2:172.21.15.169:3300/0 conn(0x7f2cc40d2360 0x7f2cc40d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 -- 172.21.15.22:0/1250895857 >> v1:172.21.15.22:6789/0 conn(0x7f2cc40cf3a0 legacy=0x7f2cc40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 -- 172.21.15.22:0/1250895857 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40d2e40 con 0x7f2cc40d1f30 2023-06-23T18:04:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cc895b700 1 --2- 172.21.15.22:0/1250895857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40d0020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2cc408bde0 tx=0x7f2cc0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d07916a839a4a2f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cbbfff700 1 -- 172.21.15.22:0/1250895857 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0011830 con 0x7f2cc40d1f30 2023-06-23T18:04:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cbbfff700 1 -- 172.21.15.22:0/1250895857 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0011990 con 0x7f2cc40d1f30 2023-06-23T18:04:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cbbfff700 1 -- 172.21.15.22:0/1250895857 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0011d40 con 0x7f2cc40d1f30 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- 172.21.15.22:0/1250895857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 msgr2=0x7f2cc40d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1250895857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40d0020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2cc408bde0 tx=0x7f2cc0009b30 comp rx=0 tx=0).stop 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- 172.21.15.22:0/1250895857 shutdown_connections 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1250895857 >> v2:172.21.15.169:3300/0 conn(0x7f2cc40d2360 0x7f2cc40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1250895857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40d0020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- 172.21.15.22:0/1250895857 >> 172.21.15.22:0/1250895857 conn(0x7f2cc406a280 msgr2=0x7f2cc409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- 172.21.15.22:0/1250895857 shutdown_connections 2023-06-23T18:04:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.081+0000 7f2cca15e700 1 -- 172.21.15.22:0/1250895857 wait complete. 2023-06-23T18:04:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 Processor -- start 2023-06-23T18:04:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 -- start start 2023-06-23T18:04:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cc40cf3a0 0x7f2cc40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 0x7f2cc40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40afee0 con 0x7f2cc40d1f30 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40b01c0 con 0x7f2cc40d2360 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cca15e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40b04a0 con 0x7f2cc40cf3a0 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49368/0 (socket says 172.21.15.22:49368) 2023-06-23T18:04:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 -- 172.21.15.22:0/1609020747 learned_addr learned my addr 172.21.15.22:0/1609020747 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc915c700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cc40cf3a0 0x7f2cc40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc995d700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 0x7f2cc40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 -- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cc40cf3a0 msgr2=0x7f2cc40c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cc40cf3a0 0x7f2cc40c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 -- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 msgr2=0x7f2cc40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 0x7f2cc40c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 -- 172.21.15.22:0/1609020747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc00094b0 con 0x7f2cc40d1f30 2023-06-23T18:04:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc995d700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 0x7f2cc40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cc895b700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00090b0 tx=0x7f2cc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004480 con 0x7f2cc40d1f30 2023-06-23T18:04:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.085+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc001f070 con 0x7f2cc40d1f30 2023-06-23T18:04:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc40b0780 con 0x7f2cc40d1f30 2023-06-23T18:04:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0003900 con 0x7f2cc40d1f30 2023-06-23T18:04:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc40c8820 con 0x7f2cc40d1f30 2023-06-23T18:04:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc404e4c0 con 0x7f2cc40d1f30 2023-06-23T18:04:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2cc0021030 con 0x7f2cc40d1f30 2023-06-23T18:04:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.089+0000 7f2cb9ffb700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 0x7f2ca80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.093+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2cc0085120 con 0x7f2cc40d1f30 2023-06-23T18:04:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.093+0000 7f2cc915c700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 0x7f2ca80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.093+0000 7f2cc915c700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 0x7f2ca80687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007fd0 tx=0x7f2cb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.097+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2cc0052330 con 0x7f2cc40d1f30 2023-06-23T18:04:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.261+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2cc40b18a0 con 0x7f2cc40d1f30 2023-06-23T18:04:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.261+0000 7f2cb9ffb700 1 -- 172.21.15.22:0/1609020747 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2cc0055950 con 0x7f2cc40d1f30 2023-06-23T18:04:25.273 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 msgr2=0x7f2ca80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 0x7f2ca80687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007fd0 tx=0x7f2cb4009040 comp rx=0 tx=0).stop 2023-06-23T18:04:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 msgr2=0x7f2cc40cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00090b0 tx=0x7f2cc0004770 comp rx=0 tx=0).stop 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 shutdown_connections 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca80662f0 0x7f2ca80687a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cc40d2360 0x7f2cc40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc40d1f30 0x7f2cc40cc0b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 --2- 172.21.15.22:0/1609020747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cc40cf3a0 0x7f2cc40c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 >> 172.21.15.22:0/1609020747 conn(0x7f2cc406a280 msgr2=0x7f2cc40b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 shutdown_connections 2023-06-23T18:04:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.265+0000 7f2cca15e700 1 -- 172.21.15.22:0/1609020747 wait complete. 2023-06-23T18:04:25.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:25.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.401+0000 7f3d07fff700 1 Processor -- start 2023-06-23T18:04:25.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.401+0000 7f3d07fff700 1 -- start start 2023-06-23T18:04:25.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d080d1f30 0x7f3d080d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d080ccb30 con 0x7f3d080d2360 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080cfcb0 con 0x7f3d080d1f30 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080d2880 con 0x7f3d080cf3a0 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d080d1f30 0x7f3d080d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d080d1f30 0x7f3d080d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49382/0 (socket says 172.21.15.22:49382) 2023-06-23T18:04:25.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 -- 172.21.15.22:0/1895788975 learned_addr learned my addr 172.21.15.22:0/1895788975 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:25.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d06ffd700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 -- 172.21.15.22:0/1895788975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 msgr2=0x7f3d080cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:25.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 -- 172.21.15.22:0/1895788975 >> v1:172.21.15.169:6789/0 conn(0x7f3d080d2360 legacy=0x7f3d0815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 -- 172.21.15.22:0/1895788975 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080d2b60 con 0x7f3d080d1f30 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3cfffff700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080d0020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009910 tx=0x7f3cf0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdee4e3978ed6cd8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d067fc700 1 -- 172.21.15.22:0/1895788975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0011830 con 0x7f3d080d1f30 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d067fc700 1 -- 172.21.15.22:0/1895788975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf0011990 con 0x7f3d080d1f30 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d067fc700 1 -- 172.21.15.22:0/1895788975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0011d40 con 0x7f3d080d1f30 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- 172.21.15.22:0/1895788975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 msgr2=0x7f3d080d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080d0020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009910 tx=0x7f3cf0009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:25.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- 172.21.15.22:0/1895788975 shutdown_connections 2023-06-23T18:04:25.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080d0020 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 --2- 172.21.15.22:0/1895788975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- 172.21.15.22:0/1895788975 >> 172.21.15.22:0/1895788975 conn(0x7f3d0806a280 msgr2=0x7f3d0809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- 172.21.15.22:0/1895788975 shutdown_connections 2023-06-23T18:04:25.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.405+0000 7f3d07fff700 1 -- 172.21.15.22:0/1895788975 wait complete. 2023-06-23T18:04:25.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.409+0000 7f3d07fff700 1 Processor -- start 2023-06-23T18:04:25.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.409+0000 7f3d07fff700 1 -- start start 2023-06-23T18:04:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 0x7f3d080c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0815e0a0 con 0x7f3d080d1f30 2023-06-23T18:04:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080b7480 con 0x7f3d080d2360 2023-06-23T18:04:25.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080c9d40 con 0x7f3d080cf3a0 2023-06-23T18:04:25.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49392/0 (socket says 172.21.15.22:49392) 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 -- 172.21.15.22:0/2007010222 learned_addr learned my addr 172.21.15.22:0/2007010222 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d06ffd700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d077fe700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 0x7f3d080c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 -- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 msgr2=0x7f3d080b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 -- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 msgr2=0x7f3d080c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 0x7f3d080c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 -- 172.21.15.22:0/2007010222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009570 con 0x7f3d080d1f30 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d077fe700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 0x7f3d080c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cfffff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3d08066d40 tx=0x7f3cf0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0004510 con 0x7f3d080d1f30 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf00036c0 con 0x7f3d080d1f30 2023-06-23T18:04:25.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d080b5940 con 0x7f3d080d1f30 2023-06-23T18:04:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0003a70 con 0x7f3d080d1f30 2023-06-23T18:04:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d080b5f00 con 0x7f3d080d1f30 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3cf0003bd0 con 0x7f3d080d1f30 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.413+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0815c290 con 0x7f3d080d1f30 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.417+0000 7f3cff7fe700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 0x7f3ce80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.417+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3cf0021030 con 0x7f3d080d1f30 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.417+0000 7f3d06ffd700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 0x7f3ce80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.417+0000 7f3d06ffd700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 0x7f3ce80687d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009be0 tx=0x7f3cf80091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.425+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3d0815c290 con 0x7f3d080d1f30 2023-06-23T18:04:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.585+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d0815c290 con 0x7f3d080d1f30 2023-06-23T18:04:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.585+0000 7f3cff7fe700 1 -- 172.21.15.22:0/2007010222 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d0815c290 con 0x7f3d080d1f30 2023-06-23T18:04:25.598 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:25.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 msgr2=0x7f3ce80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 0x7f3ce80687d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009be0 tx=0x7f3cf80091c0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 msgr2=0x7f3d080b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3d08066d40 tx=0x7f3cf0004770 comp rx=0 tx=0).stop 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 shutdown_connections 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3ce8066320 0x7f3ce80687d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d080d2360 0x7f3d080c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d080d1f30 0x7f3d080b6f40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 --2- 172.21.15.22:0/2007010222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d080cf3a0 0x7f3d080b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 >> 172.21.15.22:0/2007010222 conn(0x7f3d0806a280 msgr2=0x7f3d0815b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 shutdown_connections 2023-06-23T18:04:25.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.589+0000 7f3d07fff700 1 -- 172.21.15.22:0/2007010222 wait complete. 2023-06-23T18:04:25.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:25.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 Processor -- start 2023-06-23T18:04:25.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 -- start start 2023-06-23T18:04:25.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 0x7f1f080d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080ccb30 con 0x7f1f080d2360 2023-06-23T18:04:25.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080d2880 con 0x7f1f080cf3a0 2023-06-23T18:04:25.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f07fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080d2b60 con 0x7f1f080d1f30 2023-06-23T18:04:25.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f077fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1f080d2360 0x7f1f080ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34008/0 (socket says 172.21.15.22:34008) 2023-06-23T18:04:25.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f077fe700 1 -- 172.21.15.22:0/2008052400 learned_addr learned my addr 172.21.15.22:0/2008052400 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:25.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f067fc700 1 --2- 172.21.15.22:0/2008052400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 0x7f1f080d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960278234 0 0) 0x7f1f080ccb30 con 0x7f1f080d2360 2023-06-23T18:04:25.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef00035d0 con 0x7f1f080d2360 2023-06-23T18:04:25.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109440646 0 0) 0x7f1f080d2880 con 0x7f1f080cf3a0 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f080ccb30 con 0x7f1f080cf3a0 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f067fc700 1 -- 172.21.15.22:0/2008052400 >> v1:172.21.15.169:6789/0 conn(0x7f1f080cf3a0 legacy=0x7f1f080cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f067fc700 1 -- 172.21.15.22:0/2008052400 >> v1:172.21.15.22:6789/0 conn(0x7f1f080d2360 legacy=0x7f1f080ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.729+0000 7f1f067fc700 1 -- 172.21.15.22:0/2008052400 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080d2e40 con 0x7f1f080d1f30 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f067fc700 1 --2- 172.21.15.22:0/2008052400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 0x7f1f080d0020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009880 tx=0x7f1efc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed751b5e90df0a86 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc011830 con 0x7f1f080d1f30 2023-06-23T18:04:25.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc011990 con 0x7f1f080d1f30 2023-06-23T18:04:25.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f05ffb700 1 -- 172.21.15.22:0/2008052400 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc011d20 con 0x7f1f080d1f30 2023-06-23T18:04:25.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- 172.21.15.22:0/2008052400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 msgr2=0x7f1f080d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2008052400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 0x7f1f080d0020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009880 tx=0x7f1efc009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:25.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- 172.21.15.22:0/2008052400 shutdown_connections 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2008052400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d1f30 0x7f1f080d0020 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- 172.21.15.22:0/2008052400 >> 172.21.15.22:0/2008052400 conn(0x7f1f0806a280 msgr2=0x7f1f0809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- 172.21.15.22:0/2008052400 shutdown_connections 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- 172.21.15.22:0/2008052400 wait complete. 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 Processor -- start 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- start start 2023-06-23T18:04:25.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f080cf3a0 0x7f1f080b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d2360 0x7f1f0815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0815ef30 con 0x7f1f080d1f30 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f24f0 con 0x7f1f080cf3a0 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f07fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f27d0 con 0x7f1f080d2360 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49396/0 (socket says 172.21.15.22:49396) 2023-06-23T18:04:25.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.733+0000 7f1f067fc700 1 -- 172.21.15.22:0/2952470471 learned_addr learned my addr 172.21.15.22:0/2952470471 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:25.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f077fe700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d2360 0x7f1f0815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f06ffd700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f080cf3a0 0x7f1f080b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 -- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d2360 msgr2=0x7f1f0815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d2360 0x7f1f0815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 -- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f080cf3a0 msgr2=0x7f1f080b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f080cf3a0 0x7f1f080b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 -- 172.21.15.22:0/2952470471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0094b0 con 0x7f1f080d1f30 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f067fc700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1efc013040 tx=0x7f1efc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc01a8c0 con 0x7f1f080d1f30 2023-06-23T18:04:25.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f081f2ad0 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081f3090 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc01e070 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc01ac20 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee8005310 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.737+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1efc02d450 con 0x7f1f080d1f30 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.741+0000 7f1eeffff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 0x7f1ef40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:25.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.741+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1efc084f50 con 0x7f1f080d1f30 2023-06-23T18:04:25.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.741+0000 7f1f06ffd700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 0x7f1ef40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:25.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.741+0000 7f1f06ffd700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 0x7f1ef40687a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8001230 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:25.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.745+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1efc0b5700 con 0x7f1f080d1f30 2023-06-23T18:04:25.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.909+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ee8005b40 con 0x7f1f080d1f30 2023-06-23T18:04:25.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.909+0000 7f1eeffff700 1 -- 172.21.15.22:0/2952470471 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1efc051ff0 con 0x7f1f080d1f30 2023-06-23T18:04:25.922 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:25.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 msgr2=0x7f1ef40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 0x7f1ef40687a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8001230 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2023-06-23T18:04:25.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 msgr2=0x7f1f0815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:25.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1efc013040 tx=0x7f1efc004770 comp rx=0 tx=0).stop 2023-06-23T18:04:25.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 shutdown_connections 2023-06-23T18:04:25.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f080d2360 0x7f1f0815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f080d1f30 0x7f1f0815a630 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1ef40662f0 0x7f1ef40687a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 --2- 172.21.15.22:0/2952470471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f080cf3a0 0x7f1f080b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:25.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 >> 172.21.15.22:0/2952470471 conn(0x7f1f0806a280 msgr2=0x7f1f0806b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:25.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 shutdown_connections 2023-06-23T18:04:25.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:25.913+0000 7f1f07fff700 1 -- 172.21.15.22:0/2952470471 wait complete. 2023-06-23T18:04:25.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:26.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 Processor -- start 2023-06-23T18:04:26.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 -- start start 2023-06-23T18:04:26.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f37f80cf3a0 0x7f37f80cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f80ccb30 con 0x7f37f80d1f30 2023-06-23T18:04:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f80cfcb0 con 0x7f37f80d0730 2023-06-23T18:04:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37fd532700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f80d2880 con 0x7f37f80cf3a0 2023-06-23T18:04:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f37f80cf3a0 0x7f37f80cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f77fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f37f80d1f30 0x7f37f80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34018/0 (socket says 172.21.15.22:34018) 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f77fe700 1 -- 172.21.15.22:0/3703279254 learned_addr learned my addr 172.21.15.22:0/3703279254 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f7fff700 1 -- 172.21.15.22:0/3703279254 >> v1:172.21.15.169:6789/0 conn(0x7f37f80d0730 legacy=0x7f37f80ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f7fff700 1 -- 172.21.15.22:0/3703279254 >> v1:172.21.15.22:6789/0 conn(0x7f37f80d1f30 legacy=0x7f37f80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f6ffd700 1 -- 172.21.15.22:0/3703279254 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701375710 0 0) 0x7f37f80ccb30 con 0x7f37f80d1f30 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f7fff700 1 -- 172.21.15.22:0/3703279254 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80d2b60 con 0x7f37f80cf3a0 2023-06-23T18:04:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f7fff700 1 --2- 172.21.15.22:0/3703279254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80cf3a0 0x7f37f80cf770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009840 tx=0x7f37e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8792a23139985a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f6ffd700 1 -- 172.21.15.22:0/3703279254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8011830 con 0x7f37f80cf3a0 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f6ffd700 1 -- 172.21.15.22:0/3703279254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8011990 con 0x7f37f80cf3a0 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.053+0000 7f37f6ffd700 1 -- 172.21.15.22:0/3703279254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8011d20 con 0x7f37f80cf3a0 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- 172.21.15.22:0/3703279254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80cf3a0 msgr2=0x7f37f80cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 --2- 172.21.15.22:0/3703279254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80cf3a0 0x7f37f80cf770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009840 tx=0x7f37e8009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- 172.21.15.22:0/3703279254 shutdown_connections 2023-06-23T18:04:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 --2- 172.21.15.22:0/3703279254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80cf3a0 0x7f37f80cf770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- 172.21.15.22:0/3703279254 >> 172.21.15.22:0/3703279254 conn(0x7f37f806a280 msgr2=0x7f37f809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- 172.21.15.22:0/3703279254 shutdown_connections 2023-06-23T18:04:26.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- 172.21.15.22:0/3703279254 wait complete. 2023-06-23T18:04:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 Processor -- start 2023-06-23T18:04:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.057+0000 7f37fd532700 1 -- start start 2023-06-23T18:04:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f80cf3a0 0x7f37f81f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f80d1f30 0x7f37f815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f81f1bc0 con 0x7f37f80d0730 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f815d5b0 con 0x7f37f80d1f30 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f81f1e10 con 0x7f37f80cf3a0 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49418/0 (socket says 172.21.15.22:49418) 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 -- 172.21.15.22:0/3500476357 learned_addr learned my addr 172.21.15.22:0/3500476357 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fcd31700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f80d1f30 0x7f37f815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f7fff700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f80cf3a0 0x7f37f81f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 -- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f80cf3a0 msgr2=0x7f37f81f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f80cf3a0 0x7f37f81f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 -- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f80d1f30 msgr2=0x7f37f815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f80d1f30 0x7f37f815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 -- 172.21.15.22:0/3500476357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8009450 con 0x7f37f80d0730 2023-06-23T18:04:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f77fe700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c4a0 tx=0x7f37ec00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec004ca0 con 0x7f37f80d0730 2023-06-23T18:04:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec004e00 con 0x7f37f80d0730 2023-06-23T18:04:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec007310 con 0x7f37f80d0730 2023-06-23T18:04:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f81f20f0 con 0x7f37f80d0730 2023-06-23T18:04:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.061+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f8157ea0 con 0x7f37f80d0730 2023-06-23T18:04:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f81f2610 con 0x7f37f80d0730 2023-06-23T18:04:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f37ec007470 con 0x7f37f80d0730 2023-06-23T18:04:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37f4ff9700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 0x7f37e40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37f7fff700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 0x7f37e40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f37ec083be0 con 0x7f37f80d0730 2023-06-23T18:04:26.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.065+0000 7f37f7fff700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 0x7f37e40687a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e80090b0 tx=0x7f37e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.069+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f37f81f2610 con 0x7f37f80d0730 2023-06-23T18:04:26.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.233+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f37f81f2610 con 0x7f37f80d0730 2023-06-23T18:04:26.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.233+0000 7f37f4ff9700 1 -- 172.21.15.22:0/3500476357 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f37f81f2610 con 0x7f37f80d0730 2023-06-23T18:04:26.245 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:26.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 msgr2=0x7f37e40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 0x7f37e40687a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e80090b0 tx=0x7f37e8008030 comp rx=0 tx=0).stop 2023-06-23T18:04:26.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 msgr2=0x7f37f8157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c4a0 tx=0x7f37ec00c7b0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 shutdown_connections 2023-06-23T18:04:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f37e40662f0 0x7f37e40687a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f80d1f30 0x7f37f815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f80d0730 0x7f37f8157960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 --2- 172.21.15.22:0/3500476357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37f80cf3a0 0x7f37f81f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 >> 172.21.15.22:0/3500476357 conn(0x7f37f806a280 msgr2=0x7f37f80cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 shutdown_connections 2023-06-23T18:04:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.237+0000 7f37fd532700 1 -- 172.21.15.22:0/3500476357 wait complete. 2023-06-23T18:04:26.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 Processor -- start 2023-06-23T18:04:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 -- start start 2023-06-23T18:04:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb180afa90 0x7feb180afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb1815d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb18071940 con 0x7feb1806f660 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1815a020 con 0x7feb180afa90 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.365+0000 7feb1ed4d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb180b03a0 con 0x7feb180714c0 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1d54a700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7feb1806f660 0x7feb18070db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:37324/0 (socket says 172.21.15.22:37324) 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb1815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1d54a700 1 -- 172.21.15.22:0/2655934368 learned_addr learned my addr 172.21.15.22:0/2655934368 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1dd4b700 1 --2- 172.21.15.22:0/2655934368 >> v2:172.21.15.169:3300/0 conn(0x7feb180afa90 0x7feb180afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1cd49700 1 -- 172.21.15.22:0/2655934368 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322277981 0 0) 0x7feb18071940 con 0x7feb1806f660 2023-06-23T18:04:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1cd49700 1 -- 172.21.15.22:0/2655934368 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb040035d0 con 0x7feb1806f660 2023-06-23T18:04:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 -- 172.21.15.22:0/2655934368 >> v2:172.21.15.169:3300/0 conn(0x7feb180afa90 msgr2=0x7feb180afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 --2- 172.21.15.22:0/2655934368 >> v2:172.21.15.169:3300/0 conn(0x7feb180afa90 0x7feb180afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 -- 172.21.15.22:0/2655934368 >> v1:172.21.15.169:6789/0 conn(0x7feb1806f660 legacy=0x7feb18070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 -- 172.21.15.22:0/2655934368 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1815a210 con 0x7feb180714c0 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1e54c700 1 --2- 172.21.15.22:0/2655934368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb1815d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb10009850 tx=0x7feb10009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38997c5be701c3f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1cd49700 1 -- 172.21.15.22:0/2655934368 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10011830 con 0x7feb180714c0 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1cd49700 1 -- 172.21.15.22:0/2655934368 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10011990 con 0x7feb180714c0 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1cd49700 1 -- 172.21.15.22:0/2655934368 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10011d40 con 0x7feb180714c0 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 -- 172.21.15.22:0/2655934368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 msgr2=0x7feb1815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/2655934368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb1815d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb10009850 tx=0x7feb10009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 -- 172.21.15.22:0/2655934368 shutdown_connections 2023-06-23T18:04:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/2655934368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb1815d760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/2655934368 >> v2:172.21.15.169:3300/0 conn(0x7feb180afa90 0x7feb180afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 -- 172.21.15.22:0/2655934368 >> 172.21.15.22:0/2655934368 conn(0x7feb1806a5f0 msgr2=0x7feb180caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 -- 172.21.15.22:0/2655934368 shutdown_connections 2023-06-23T18:04:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.369+0000 7feb1ed4d700 1 -- 172.21.15.22:0/2655934368 wait complete. 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 Processor -- start 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- start start 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb1806f660 0x7feb180b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb180b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb181f2480 con 0x7feb180afa90 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb181ef3b0 con 0x7feb1806f660 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb181ef690 con 0x7feb180714c0 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1e54c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1e54c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54264/0 (socket says 172.21.15.22:54264) 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1e54c700 1 -- 172.21.15.22:0/3356468850 learned_addr learned my addr 172.21.15.22:0/3356468850 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb180b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1dd4b700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb1806f660 0x7feb180b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 -- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb1806f660 msgr2=0x7feb180b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb1806f660 0x7feb180b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 -- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 msgr2=0x7feb180c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 -- 172.21.15.22:0/3356468850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100094b0 con 0x7feb180714c0 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1e54c700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1d54a700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb180b0e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb1400caa0 tx=0x7feb1400cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14007480 con 0x7feb180714c0 2023-06-23T18:04:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140075e0 con 0x7feb180714c0 2023-06-23T18:04:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14007950 con 0x7feb180714c0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb181ef970 con 0x7feb180714c0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.373+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb180b2120 con 0x7feb180714c0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb181f2630 con 0x7feb180714c0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7feb1401b3b0 con 0x7feb180714c0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb0e7fc700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 0x7feafc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb1dd4b700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 0x7feafc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb14083980 con 0x7feb180714c0 2023-06-23T18:04:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.377+0000 7feb1dd4b700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 0x7feafc0687a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb180c4c80 tx=0x7feb08009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.385+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7feb140547c0 con 0x7feb180714c0 2023-06-23T18:04:26.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.557+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb181f25f0 con 0x7feb180714c0 2023-06-23T18:04:26.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.561+0000 7feb0e7fc700 1 -- 172.21.15.22:0/3356468850 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20204 (secure 0 0 0) 0x7feb14054350 con 0x7feb180714c0 2023-06-23T18:04:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:26.572 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:26.411675+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:26.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 msgr2=0x7feafc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 0x7feafc0687a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb180c4c80 tx=0x7feb08009240 comp rx=0 tx=0).stop 2023-06-23T18:04:26.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 msgr2=0x7feb180b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb180b0e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb1400caa0 tx=0x7feb1400cad0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 shutdown_connections 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb180afa90 0x7feb180c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb180714c0 0x7feb180b0e40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7feafc0662f0 0x7feafc0687a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 --2- 172.21.15.22:0/3356468850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb1806f660 0x7feb180b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 >> 172.21.15.22:0/3356468850 conn(0x7feb1806a5f0 msgr2=0x7feb18159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 shutdown_connections 2023-06-23T18:04:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.565+0000 7feb1ed4d700 1 -- 172.21.15.22:0/3356468850 wait complete. 2023-06-23T18:04:26.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:26.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:26.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 Processor -- start 2023-06-23T18:04:26.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 -- start start 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6c00cf3a0 0x7fc6c00cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00ccb30 con 0x7fc6c00d0730 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cfcb0 con 0x7fc6c00d1f30 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bf59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00d2880 con 0x7fc6c00cf3a0 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bed9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc6c00d0730 0x7fc6c00ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58304/0 (socket says 172.21.15.22:58304) 2023-06-23T18:04:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.693+0000 7fc6bed9d700 1 -- 172.21.15.22:0/1690178488 learned_addr learned my addr 172.21.15.22:0/1690178488 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:26.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6be59c700 1 --2- 172.21.15.22:0/1690178488 >> v2:172.21.15.169:3300/0 conn(0x7fc6c00cf3a0 0x7fc6c00cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425875932 0 0) 0x7fc6c00ccb30 con 0x7fc6c00d0730 2023-06-23T18:04:26.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ac0035d0 con 0x7fc6c00d0730 2023-06-23T18:04:26.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1971316751 0 0) 0x7fc6ac0035d0 con 0x7fc6c00d0730 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 >> v2:172.21.15.169:3300/0 conn(0x7fc6c00cf3a0 msgr2=0x7fc6c00cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 --2- 172.21.15.22:0/1690178488 >> v2:172.21.15.169:3300/0 conn(0x7fc6c00cf3a0 0x7fc6c00cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 >> v1:172.21.15.169:6789/0 conn(0x7fc6c00d1f30 legacy=0x7fc6c00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00d2b60 con 0x7fc6c00d0730 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fc6b80019f0 con 0x7fc6c00d0730 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 >> v1:172.21.15.22:6789/0 conn(0x7fc6c00d0730 legacy=0x7fc6c00ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ac003a20 0x7fc6ac003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 0x7fc6ac0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac0050d0 0x7fc6ac00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00d2880 con 0x7fc6ac004410 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00bf70 con 0x7fc6ac0050d0 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00c130 con 0x7fc6ac003a20 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 0x7fc6ac0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6be59c700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ac003a20 0x7fc6ac003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bed9d700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac0050d0 0x7fc6ac00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ac003a20 msgr2=0x7fc6ac003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ac003a20 0x7fc6ac003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac0050d0 msgr2=0x7fc6ac00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac0050d0 0x7fc6ac00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1690178488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0035d0 con 0x7fc6ac004410 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bed9d700 1 -- 172.21.15.22:0/1690178488 reap_dead start 2023-06-23T18:04:26.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 0x7fc6ac0048d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80018f0 tx=0x7fc6ac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff2b62b6f635aaf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8001390 con 0x7fc6ac004410 2023-06-23T18:04:26.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a80052f0 con 0x7fc6ac004410 2023-06-23T18:04:26.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bdd9b700 1 -- 172.21.15.22:0/1690178488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80036a0 con 0x7fc6ac004410 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 msgr2=0x7fc6ac0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 0x7fc6ac0048d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80018f0 tx=0x7fc6ac0058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1690178488 shutdown_connections 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1690178488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6ac004410 0x7fc6ac0048d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1690178488 >> 172.21.15.22:0/1690178488 conn(0x7fc6c006a280 msgr2=0x7fc6c009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1690178488 shutdown_connections 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.697+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1690178488 wait complete. 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 Processor -- start 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- start start 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac004410 0x7fc6c00c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c015dda0 0x7fc6c00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015e2b0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015e590 con 0x7fc6ac004410 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015e870 con 0x7fc6c015dda0 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54276/0 (socket says 172.21.15.22:54276) 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1033247711 learned_addr learned my addr 172.21.15.22:0/1033247711 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6be59c700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac004410 0x7fc6c00c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bed9d700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c015dda0 0x7fc6c00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c015dda0 msgr2=0x7fc6c00cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c015dda0 0x7fc6c00cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac004410 msgr2=0x7fc6c00c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac004410 0x7fc6c00c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 -- 172.21.15.22:0/1033247711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c015eb50 con 0x7fc6c0150ea0 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b7fff700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8003f90 tx=0x7fc6a8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800c490 con 0x7fc6c0150ea0 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800c5f0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c00c8480 con 0x7fc6c0150ea0 2023-06-23T18:04:26.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800c980 con 0x7fc6c0150ea0 2023-06-23T18:04:26.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.701+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c00ce580 con 0x7fc6c0150ea0 2023-06-23T18:04:26.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c014e2a0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc6a800ccf0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6b6ffd700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 0x7fc6a00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:26.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc6a8085300 con 0x7fc6c0150ea0 2023-06-23T18:04:26.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6be59c700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 0x7fc6a00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:26.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.705+0000 7fc6be59c700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 0x7fc6a00687d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0063530 tx=0x7fc6b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.713+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc6c014e2a0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.877+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6c014e2a0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6b6ffd700 1 -- 172.21.15.22:0/1033247711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6c014e2a0 con 0x7fc6c0150ea0 2023-06-23T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:26.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 msgr2=0x7fc6a00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 0x7fc6a00687d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0063530 tx=0x7fc6b0008040 comp rx=0 tx=0).stop 2023-06-23T18:04:26.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 msgr2=0x7fc6c015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:26.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8003f90 tx=0x7fc6a8001230 comp rx=0 tx=0).stop 2023-06-23T18:04:26.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 shutdown_connections 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c015dda0 0x7fc6c00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0150ea0 0x7fc6c015d860 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc6a0066320 0x7fc6a00687d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 --2- 172.21.15.22:0/1033247711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ac004410 0x7fc6c00c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 >> 172.21.15.22:0/1033247711 conn(0x7fc6c006a280 msgr2=0x7fc6c00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 shutdown_connections 2023-06-23T18:04:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:26.881+0000 7fc6bf59e700 1 -- 172.21.15.22:0/1033247711 wait complete. 2023-06-23T18:04:26.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 Processor -- start 2023-06-23T18:04:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 -- start start 2023-06-23T18:04:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 0x7f4c74070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740cbc50 con 0x7f4c740ce7e0 2023-06-23T18:04:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7415c470 con 0x7f4c7406e140 2023-06-23T18:04:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c7b9f7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415d420 con 0x7f4c74152ea0 2023-06-23T18:04:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79793700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79f94700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 0x7f4c74070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c78f92700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4c740ce7e0 0x7f4c7406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:37344/0 (socket says 172.21.15.22:37344) 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c78f92700 1 -- 172.21.15.22:0/3034314974 learned_addr learned my addr 172.21.15.22:0/3034314974 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79793700 1 -- 172.21.15.22:0/3034314974 >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 msgr2=0x7f4c74070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79793700 1 --2- 172.21.15.22:0/3034314974 >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 0x7f4c74070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79793700 1 -- 172.21.15.22:0/3034314974 >> v1:172.21.15.169:6789/0 conn(0x7f4c740ce7e0 legacy=0x7f4c7406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79793700 1 -- 172.21.15.22:0/3034314974 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740cf010 con 0x7f4c74152ea0 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.017+0000 7f4c79f94700 1 --2- 172.21.15.22:0/3034314974 >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 0x7f4c74070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c79793700 1 --2- 172.21.15.22:0/3034314974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74160530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009850 tx=0x7f4c70009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d535ce222d4edc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c6bfff700 1 -- 172.21.15.22:0/3034314974 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70011830 con 0x7f4c74152ea0 2023-06-23T18:04:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c6bfff700 1 -- 172.21.15.22:0/3034314974 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c70011990 con 0x7f4c74152ea0 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c6bfff700 1 -- 172.21.15.22:0/3034314974 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70011d40 con 0x7f4c74152ea0 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/3034314974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 msgr2=0x7f4c74160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/3034314974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74160530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009850 tx=0x7f4c70009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/3034314974 shutdown_connections 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/3034314974 >> v2:172.21.15.169:3300/0 conn(0x7f4c7406e140 0x7f4c74070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/3034314974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74160530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/3034314974 >> 172.21.15.22:0/3034314974 conn(0x7f4c7406e980 msgr2=0x7f4c7406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/3034314974 shutdown_connections 2023-06-23T18:04:27.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/3034314974 wait complete. 2023-06-23T18:04:27.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 Processor -- start 2023-06-23T18:04:27.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.021+0000 7f4c7b9f7700 1 -- start start 2023-06-23T18:04:27.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 0x7f4c740cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740b2fd0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740b32b0 con 0x7f4c7406e140 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740b3590 con 0x7f4c74152ea0 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54290/0 (socket says 172.21.15.22:54290) 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 -- 172.21.15.22:0/2799605517 learned_addr learned my addr 172.21.15.22:0/2799605517 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c79f94700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c79793700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 0x7f4c740cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 -- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 msgr2=0x7f4c74147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74147c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 -- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 msgr2=0x7f4c740cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 0x7f4c740cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 -- 172.21.15.22:0/2799605517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009a70 con 0x7f4c740ce7e0 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c79793700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 0x7f4c740cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c78f92700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c700 tx=0x7f4c6400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64005d90 con 0x7f4c740ce7e0 2023-06-23T18:04:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640070d0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700094b0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64007460 con 0x7f4c740ce7e0 2023-06-23T18:04:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c740b3b50 con 0x7f4c740ce7e0 2023-06-23T18:04:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.025+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74060eb0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.029+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4c640075c0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.029+0000 7f4c69ffb700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 0x7f4c580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.029+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4c64083540 con 0x7f4c740ce7e0 2023-06-23T18:04:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.029+0000 7f4c79793700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 0x7f4c580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.029+0000 7f4c79793700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 0x7f4c580687a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000c00 tx=0x7f4c70008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.033+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4c640506a0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c7404dc70 con 0x7f4c740ce7e0 2023-06-23T18:04:27.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c69ffb700 1 -- 172.21.15.22:0/2799605517 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c64053cc0 con 0x7f4c740ce7e0 2023-06-23T18:04:27.208 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:27.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 msgr2=0x7f4c580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 0x7f4c580687a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000c00 tx=0x7f4c70008030 comp rx=0 tx=0).stop 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 msgr2=0x7f4c74147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c700 tx=0x7f4c6400cac0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 shutdown_connections 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c74152ea0 0x7f4c74147c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c740ce7e0 0x7f4c74147720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4c580662f0 0x7f4c580687a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 --2- 172.21.15.22:0/2799605517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c7406e140 0x7f4c740cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 >> 172.21.15.22:0/2799605517 conn(0x7f4c7406e980 msgr2=0x7f4c7406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 shutdown_connections 2023-06-23T18:04:27.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.197+0000 7f4c7b9f7700 1 -- 172.21.15.22:0/2799605517 wait complete. 2023-06-23T18:04:27.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:27.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 Processor -- start 2023-06-23T18:04:27.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 -- start start 2023-06-23T18:04:27.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4dec0cf3a0 0x7f4dec0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4dec0d2360 0x7f4dec0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0ccb30 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0cfcb0 con 0x7f4dec0d2360 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.333+0000 7f4df09f0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0d2880 con 0x7f4dec0d1f30 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4dec0cf3a0 0x7f4dec0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4dec0cf3a0 0x7f4dec0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54298/0 (socket says 172.21.15.22:54298) 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 -- 172.21.15.22:0/469223855 learned_addr learned my addr 172.21.15.22:0/469223855 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4debfff700 1 --2- 172.21.15.22:0/469223855 >> v2:172.21.15.169:3300/0 conn(0x7f4dec0d2360 0x7f4dec0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deaffd700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 -- 172.21.15.22:0/469223855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 msgr2=0x7f4dec0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 -- 172.21.15.22:0/469223855 >> v2:172.21.15.169:3300/0 conn(0x7f4dec0d2360 msgr2=0x7f4dec0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/469223855 >> v2:172.21.15.169:3300/0 conn(0x7f4dec0d2360 0x7f4dec0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 -- 172.21.15.22:0/469223855 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0d2b60 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec0cf770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4dec066ad0 tx=0x7f4ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d0902e5013f2ac server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4dea7fc700 1 -- 172.21.15.22:0/469223855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00e070 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4dea7fc700 1 -- 172.21.15.22:0/469223855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc00b9c0 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4dea7fc700 1 -- 172.21.15.22:0/469223855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00bd70 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 -- 172.21.15.22:0/469223855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 msgr2=0x7f4dec0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec0cf770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4dec066ad0 tx=0x7f4ddc00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 -- 172.21.15.22:0/469223855 shutdown_connections 2023-06-23T18:04:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 --2- 172.21.15.22:0/469223855 >> v2:172.21.15.169:3300/0 conn(0x7f4dec0d2360 0x7f4dec0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 --2- 172.21.15.22:0/469223855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec0cf770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 -- 172.21.15.22:0/469223855 >> 172.21.15.22:0/469223855 conn(0x7f4dec06a280 msgr2=0x7f4dec09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 -- 172.21.15.22:0/469223855 shutdown_connections 2023-06-23T18:04:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.337+0000 7f4df09f0700 1 -- 172.21.15.22:0/469223855 wait complete. 2023-06-23T18:04:27.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 Processor -- start 2023-06-23T18:04:27.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 -- start start 2023-06-23T18:04:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 0x7f4dec0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0b1730 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec158f40 con 0x7f4dec0d2360 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4df09f0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0b8060 con 0x7f4dec0d1f30 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54306/0 (socket says 172.21.15.22:54306) 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 -- 172.21.15.22:0/1299740047 learned_addr learned my addr 172.21.15.22:0/1299740047 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deaffd700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4debfff700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 0x7f4dec0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 -- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 msgr2=0x7f4dec0b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 -- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 msgr2=0x7f4dec0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 0x7f4dec0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4deb7fe700 1 -- 172.21.15.22:0/1299740047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009ad0 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.341+0000 7f4debfff700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 0x7f4dec0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:27.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4deb7fe700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dec066d40 tx=0x7f4ddc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0042d0 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc004450 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec1f2210 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc004800 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec1f1730 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4ddc01c440 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.345+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec04e4c0 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.349+0000 7f4de3fff700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 0x7f4dcc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.349+0000 7f4deaffd700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 0x7f4dcc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.349+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4ddc0859b0 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.349+0000 7f4deaffd700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 0x7f4dcc0687a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4dec15b8e0 tx=0x7f4dd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.357+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4ddc052b00 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.529+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dec06a680 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.529+0000 7f4de3fff700 1 -- 172.21.15.22:0/1299740047 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ddc056120 con 0x7f4dec0cf3a0 2023-06-23T18:04:27.540 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:27.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 msgr2=0x7f4dcc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 0x7f4dcc0687a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4dec15b8e0 tx=0x7f4dd4008040 comp rx=0 tx=0).stop 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 msgr2=0x7f4dec158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dec066d40 tx=0x7f4ddc004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 shutdown_connections 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4dcc0662f0 0x7f4dcc0687a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dec0d2360 0x7f4dec0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dec0d1f30 0x7f4dec0b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 --2- 172.21.15.22:0/1299740047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dec0cf3a0 0x7f4dec158a00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 >> 172.21.15.22:0/1299740047 conn(0x7f4dec06a280 msgr2=0x7f4dec0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 shutdown_connections 2023-06-23T18:04:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.533+0000 7f4df09f0700 1 -- 172.21.15.22:0/1299740047 wait complete. 2023-06-23T18:04:27.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:27.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 Processor -- start 2023-06-23T18:04:27.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 -- start start 2023-06-23T18:04:27.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f408c0d1f30 0x7f408c0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c0ccb30 con 0x7f408c0cf3a0 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c0d2880 con 0x7f408c0d2360 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4091ca2700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c0d2b60 con 0x7f408c0d1f30 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4090ca0700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f408c0cf3a0 0x7f408c0cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58306/0 (socket says 172.21.15.22:58306) 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f4090ca0700 1 -- 172.21.15.22:0/1308102283 learned_addr learned my addr 172.21.15.22:0/1308102283 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408bfff700 1 --2- 172.21.15.22:0/1308102283 >> v2:172.21.15.169:3300/0 conn(0x7f408c0d1f30 0x7f408c0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044094291 0 0) 0x7f408c0ccb30 con 0x7f408c0cf3a0 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40780035d0 con 0x7f408c0cf3a0 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045103513 0 0) 0x7f40780035d0 con 0x7f408c0cf3a0 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 >> v2:172.21.15.169:3300/0 conn(0x7f408c0d1f30 msgr2=0x7f408c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 --2- 172.21.15.22:0/1308102283 >> v2:172.21.15.169:3300/0 conn(0x7f408c0d1f30 0x7f408c0d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 >> v1:172.21.15.169:6789/0 conn(0x7f408c0d2360 legacy=0x7f408c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0d2e40 con 0x7f408c0cf3a0 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.665+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f4074005140 con 0x7f408c0cf3a0 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 >> v1:172.21.15.22:6789/0 conn(0x7f408c0cf3a0 legacy=0x7f408c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078003a20 0x7f4078003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 0x7f40780048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780050d0 0x7f407800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0d2b60 con 0x7f4078004410 2023-06-23T18:04:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407800bf70 con 0x7f40780050d0 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407800c130 con 0x7f4078003a20 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 0x7f40780048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4090ca0700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078003a20 0x7f4078003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f40914a1700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780050d0 0x7f407800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 -- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078003a20 msgr2=0x7f4078003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078003a20 0x7f4078003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 -- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780050d0 msgr2=0x7f407800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:27.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780050d0 0x7f407800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 -- 172.21.15.22:0/1308102283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780035d0 con 0x7f4078004410 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4090ca0700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078003a20 0x7f4078003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f40914a1700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780050d0 0x7f407800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f40914a1700 1 -- 172.21.15.22:0/1308102283 reap_dead start 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408bfff700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 0x7f40780048d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f407c009b70 tx=0x7f40780058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29933baa5c1a7ba server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00e6c0 con 0x7f4078004410 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00e820 con 0x7f4078004410 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f408b7fe700 1 -- 172.21.15.22:0/1308102283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00eb90 con 0x7f4078004410 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 msgr2=0x7f40780048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 0x7f40780048d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f407c009b70 tx=0x7f40780058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- 172.21.15.22:0/1308102283 shutdown_connections 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1308102283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4078004410 0x7f40780048d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- 172.21.15.22:0/1308102283 >> 172.21.15.22:0/1308102283 conn(0x7f408c06a280 msgr2=0x7f408c09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- 172.21.15.22:0/1308102283 shutdown_connections 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- 172.21.15.22:0/1308102283 wait complete. 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 Processor -- start 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.669+0000 7f4091ca2700 1 -- start start 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078004410 0x7f408c1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0b2460 0x7f408c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0b5c10 con 0x7f408c0b5280 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c159e30 con 0x7f408c0b2460 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0b2e10 con 0x7f4078004410 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4090ca0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078004410 0x7f408c1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54330/0 (socket says 172.21.15.22:54330) 2023-06-23T18:04:27.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 -- 172.21.15.22:0/1071729408 learned_addr learned my addr 172.21.15.22:0/1071729408 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f40914a1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0b2460 0x7f408c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 -- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078004410 msgr2=0x7f408c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078004410 0x7f408c1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 -- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0b2460 msgr2=0x7f408c0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0b2460 0x7f408c0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 -- 172.21.15.22:0/1071729408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c1e77b0 con 0x7f408c0b5280 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f408bfff700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f407c00efd0 tx=0x7f407c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00e6c0 con 0x7f408c0b5280 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c003af0 con 0x7f408c0b5280 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c0b5df0 con 0x7f408c0b5280 2023-06-23T18:04:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c003ea0 con 0x7f408c0b5280 2023-06-23T18:04:27.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c0b63b0 con 0x7f408c0b5280 2023-06-23T18:04:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.673+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c04e4c0 con 0x7f408c0b5280 2023-06-23T18:04:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.677+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f407c013470 con 0x7f408c0b5280 2023-06-23T18:04:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.677+0000 7f40897fa700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 0x7f406c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.677+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f407c0855c0 con 0x7f408c0b5280 2023-06-23T18:04:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.677+0000 7f4090ca0700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 0x7f406c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:27.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.677+0000 7f4090ca0700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 0x7f406c0687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f408c063530 tx=0x7f4074006660 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:27.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.685+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f407c052720 con 0x7f408c0b5280 2023-06-23T18:04:27.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.849+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f408c06a680 con 0x7f408c0b5280 2023-06-23T18:04:27.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f40897fa700 1 -- 172.21.15.22:0/1071729408 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f407c055d40 con 0x7f408c0b5280 2023-06-23T18:04:27.863 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:27.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 msgr2=0x7f406c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 0x7f406c0687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f408c063530 tx=0x7f4074006660 comp rx=0 tx=0).stop 2023-06-23T18:04:27.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 msgr2=0x7f408c0b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:27.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f407c00efd0 tx=0x7f407c0047e0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 shutdown_connections 2023-06-23T18:04:27.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f406c066320 0x7f406c0687d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f408c0b2460 0x7f408c0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408c0b5280 0x7f408c0b56d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 --2- 172.21.15.22:0/1071729408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4078004410 0x7f408c1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 >> 172.21.15.22:0/1071729408 conn(0x7f408c06a280 msgr2=0x7f408c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 shutdown_connections 2023-06-23T18:04:27.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.853+0000 7f4091ca2700 1 -- 172.21.15.22:0/1071729408 wait complete. 2023-06-23T18:04:27.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:27.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.985+0000 7fba3eb19700 1 Processor -- start 2023-06-23T18:04:27.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.985+0000 7fba3eb19700 1 -- start start 2023-06-23T18:04:27.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3eb19700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba30050760 0x7fba30036770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:27.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3eb19700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba30052bf0 con 0x7fba300dd280 2023-06-23T18:04:27.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3eb19700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba300528c0 con 0x7fba30036cb0 2023-06-23T18:04:27.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3eb19700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300dd650 con 0x7fba30050760 2023-06-23T18:04:27.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fba300dd280 0x7fba300dfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58314/0 (socket says 172.21.15.22:58314) 2023-06-23T18:04:28.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 -- 172.21.15.22:0/4135217926 learned_addr learned my addr 172.21.15.22:0/4135217926 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:28.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3d316700 1 --2- 172.21.15.22:0/4135217926 >> v2:172.21.15.169:3300/0 conn(0x7fba30050760 0x7fba30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14301290 0 0) 0x7fba30052bf0 con 0x7fba300dd280 2023-06-23T18:04:28.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba240035d0 con 0x7fba300dd280 2023-06-23T18:04:28.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780051987 0 0) 0x7fba240035d0 con 0x7fba300dd280 2023-06-23T18:04:28.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 >> v2:172.21.15.169:3300/0 conn(0x7fba30050760 msgr2=0x7fba30036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 --2- 172.21.15.22:0/4135217926 >> v2:172.21.15.169:3300/0 conn(0x7fba30050760 0x7fba30036770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 >> v1:172.21.15.169:6789/0 conn(0x7fba30036cb0 legacy=0x7fba30053080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300e0430 con 0x7fba300dd280 2023-06-23T18:04:28.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fba34001670 con 0x7fba300dd280 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 >> v1:172.21.15.22:6789/0 conn(0x7fba300dd280 legacy=0x7fba300dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 0x7fba24003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba24004450 0x7fba24004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24005110 0x7fba2400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300dd650 con 0x7fba24003a20 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2400c050 con 0x7fba24005110 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2400c210 con 0x7fba24004450 2023-06-23T18:04:28.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 0x7fba24003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3d316700 1 --2- 172.21.15.22:0/4135217926 >> v2:172.21.15.169:3300/0 conn(0x7fba30050760 0x7fba30036770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3e318700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24005110 0x7fba2400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3d316700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba24004450 0x7fba24004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 -- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba24004450 msgr2=0x7fba24004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba24004450 0x7fba24004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 -- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24005110 msgr2=0x7fba2400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24005110 0x7fba2400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 -- 172.21.15.22:0/4135217926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240035d0 con 0x7fba24003a20 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3e318700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24005110 0x7fba2400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3e318700 1 -- 172.21.15.22:0/4135217926 reap_dead start 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3d316700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba24004450 0x7fba24004910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3db17700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 0x7fba24003e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c320 tx=0x7fba240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cff4692431cb1b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400ce60 con 0x7fba24003a20 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba34005980 con 0x7fba24003a20 2023-06-23T18:04:28.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.989+0000 7fba3cb15700 1 -- 172.21.15.22:0/4135217926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34005d10 con 0x7fba24003a20 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 msgr2=0x7fba24003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 0x7fba24003e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c320 tx=0x7fba240058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- 172.21.15.22:0/4135217926 shutdown_connections 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 --2- 172.21.15.22:0/4135217926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba24003a20 0x7fba24003e30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- 172.21.15.22:0/4135217926 >> 172.21.15.22:0/4135217926 conn(0x7fba30051770 msgr2=0x7fba30051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- 172.21.15.22:0/4135217926 shutdown_connections 2023-06-23T18:04:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- 172.21.15.22:0/4135217926 wait complete. 2023-06-23T18:04:28.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 Processor -- start 2023-06-23T18:04:28.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.993+0000 7fba3eb19700 1 -- start start 2023-06-23T18:04:28.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24003a20 0x7fba300eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba300d1e40 0x7fba30005990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300d22b0 con 0x7fba300ee750 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300d1470 con 0x7fba24003a20 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300edd00 con 0x7fba300d1e40 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54346/0 (socket says 172.21.15.22:54346) 2023-06-23T18:04:28.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 -- 172.21.15.22:0/498692773 learned_addr learned my addr 172.21.15.22:0/498692773 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3db17700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24003a20 0x7fba300eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3e318700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba300d1e40 0x7fba30005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 -- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba300d1e40 msgr2=0x7fba30005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba300d1e40 0x7fba30005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 -- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24003a20 msgr2=0x7fba300eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24003a20 0x7fba300eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 -- 172.21.15.22:0/498692773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30047af0 con 0x7fba300ee750 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3d316700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba2c010920 tx=0x7fba2c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c014800 con 0x7fba300ee750 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c014960 con 0x7fba300ee750 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba30047dd0 con 0x7fba300ee750 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c014cf0 con 0x7fba300ee750 2023-06-23T18:04:28.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:27.997+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba300ee230 con 0x7fba300ee750 2023-06-23T18:04:28.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba30005570 con 0x7fba300ee750 2023-06-23T18:04:28.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fba2c01c400 con 0x7fba300ee750 2023-06-23T18:04:28.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba2a7fc700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 0x7fba1c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fba2c084250 con 0x7fba300ee750 2023-06-23T18:04:28.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba3db17700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 0x7fba1c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.001+0000 7fba3db17700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 0x7fba1c0687d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba3400a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.009+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fba2c051390 con 0x7fba300ee750 2023-06-23T18:04:28.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.169+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fba30000e50 con 0x7fba300ee750 2023-06-23T18:04:28.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.169+0000 7fba2a7fc700 1 -- 172.21.15.22:0/498692773 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fba2c0549b0 con 0x7fba300ee750 2023-06-23T18:04:28.183 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:28.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 msgr2=0x7fba1c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 0x7fba1c0687d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba3400a000 comp rx=0 tx=0).stop 2023-06-23T18:04:28.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 msgr2=0x7fba300d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba2c010920 tx=0x7fba2c010c30 comp rx=0 tx=0).stop 2023-06-23T18:04:28.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 shutdown_connections 2023-06-23T18:04:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba300d1e40 0x7fba30005990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba300ee750 0x7fba300d1900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fba1c066320 0x7fba1c0687d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 --2- 172.21.15.22:0/498692773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba24003a20 0x7fba300eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 >> 172.21.15.22:0/498692773 conn(0x7fba30051770 msgr2=0x7fba300dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 shutdown_connections 2023-06-23T18:04:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.173+0000 7fba3eb19700 1 -- 172.21.15.22:0/498692773 wait complete. 2023-06-23T18:04:28.285 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-06-23T18:04:28.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-06-23T18:04:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 Processor -- start 2023-06-23T18:04:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 -- start start 2023-06-23T18:04:28.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57c00b5a70 0x7f57c00cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c015a080 con 0x7f57c00cf1b0 2023-06-23T18:04:28.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00cc830 con 0x7f57c00cde70 2023-06-23T18:04:28.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c556c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00b62e0 con 0x7f57c00b5a70 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f57c00cf1b0 0x7f57c00cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58330/0 (socket says 172.21.15.22:58330) 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 -- 172.21.15.22:0/3364591712 learned_addr learned my addr 172.21.15.22:0/3364591712 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c4d6b700 1 --2- 172.21.15.22:0/3364591712 >> v2:172.21.15.169:3300/0 conn(0x7f57c00b5a70 0x7f57c00cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203566041 0 0) 0x7f57a8001110 con 0x7f57c00cf1b0 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57ac0035d0 con 0x7f57c00cf1b0 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2097301330 0 0) 0x7f57ac0035d0 con 0x7f57c00cf1b0 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 >> v2:172.21.15.169:3300/0 conn(0x7f57c00b5a70 msgr2=0x7f57c00cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 --2- 172.21.15.22:0/3364591712 >> v2:172.21.15.169:3300/0 conn(0x7f57c00b5a70 0x7f57c00cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 >> v1:172.21.15.169:6789/0 conn(0x7f57c00cde70 legacy=0x7f57c00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00cd3b0 con 0x7f57c00cf1b0 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f57a8001800 con 0x7f57c00cf1b0 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 >> v1:172.21.15.22:6789/0 conn(0x7f57c00cf1b0 legacy=0x7f57c00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac003a20 0x7f57ac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 0x7f57ac004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ac005220 0x7f57ac00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00b62e0 con 0x7f57ac004870 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac00c130 con 0x7f57ac003a20 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac00c2f0 con 0x7f57ac005220 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 0x7f57ac004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bffff700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac003a20 0x7f57ac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c4d6b700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ac005220 0x7f57ac00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 -- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ac005220 msgr2=0x7f57ac00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ac005220 0x7f57ac00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 -- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac003a20 msgr2=0x7f57ac003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac003a20 0x7f57ac003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 -- 172.21.15.22:0/3364591712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0035d0 con 0x7f57ac004870 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57c4d6b700 1 -- 172.21.15.22:0/3364591712 reap_dead start 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bffff700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac003a20 0x7f57ac003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.401+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 0x7f57ac004ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c320 tx=0x7f57ac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b676bd89816fe8e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a800cde0 con 0x7f57ac004870 2023-06-23T18:04:28.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8005980 con 0x7f57ac004870 2023-06-23T18:04:28.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57beffd700 1 -- 172.21.15.22:0/3364591712 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8005d30 con 0x7f57ac004870 2023-06-23T18:04:28.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 -- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 msgr2=0x7f57ac004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 0x7f57ac004ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c320 tx=0x7f57ac0058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 -- 172.21.15.22:0/3364591712 shutdown_connections 2023-06-23T18:04:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 --2- 172.21.15.22:0/3364591712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac004870 0x7f57ac004ce0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 -- 172.21.15.22:0/3364591712 >> 172.21.15.22:0/3364591712 conn(0x7f57c0069960 msgr2=0x7f57c0069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 -- 172.21.15.22:0/3364591712 shutdown_connections 2023-06-23T18:04:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.405+0000 7f57c556c700 1 -- 172.21.15.22:0/3364591712 wait complete. 2023-06-23T18:04:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 Processor -- start 2023-06-23T18:04:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 -- start start 2023-06-23T18:04:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac004870 0x7f57c00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c00c7370 0x7f57c00c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00b3170 con 0x7f57c00b4c10 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00b3450 con 0x7f57ac004870 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c556c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00b3730 con 0x7f57c00c7370 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54360/0 (socket says 172.21.15.22:54360) 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 -- 172.21.15.22:0/716577672 learned_addr learned my addr 172.21.15.22:0/716577672 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bffff700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac004870 0x7f57c00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57c4d6b700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c00c7370 0x7f57c00c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 -- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c00c7370 msgr2=0x7f57c00c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c00c7370 0x7f57c00c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 -- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac004870 msgr2=0x7f57c00d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac004870 0x7f57c00d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 -- 172.21.15.22:0/716577672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00b3a10 con 0x7f57c00b4c10 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.409+0000 7f57bf7fe700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00c100 tx=0x7f57a8001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007cf0 con 0x7f57c00b4c10 2023-06-23T18:04:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a80017d0 con 0x7f57c00b4c10 2023-06-23T18:04:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c00b3cf0 con 0x7f57c00b4c10 2023-06-23T18:04:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8004a90 con 0x7f57c00b4c10 2023-06-23T18:04:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00d0de0 con 0x7f57c00b4c10 2023-06-23T18:04:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c004e4c0 con 0x7f57c00b4c10 2023-06-23T18:04:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f57a8004bf0 con 0x7f57c00b4c10 2023-06-23T18:04:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 0x7f57a00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bffff700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 0x7f57a00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f57a8084bf0 con 0x7f57c00b4c10 2023-06-23T18:04:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.413+0000 7f57bffff700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 0x7f57a00687d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000f80 tx=0x7f57b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.421+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f57a8051f20 con 0x7f57c00b4c10 2023-06-23T18:04:28.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:28.646+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f57c015d510 con 0x7f57c00b4c10 2023-06-23T18:04:29.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.450+0000 7f57bcff9700 1 -- 172.21.15.22:0/716577672 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v64) v1 ==== 147+0+0 (secure 0 0 0) 0x7f57a8055540 con 0x7f57c00b4c10 2023-06-23T18:04:29.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 msgr2=0x7f57a00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 0x7f57a00687d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000f80 tx=0x7f57b8009040 comp rx=0 tx=0).stop 2023-06-23T18:04:29.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 msgr2=0x7f57c00b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00c100 tx=0x7f57a8001b90 comp rx=0 tx=0).stop 2023-06-23T18:04:29.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 shutdown_connections 2023-06-23T18:04:29.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57c00c7370 0x7f57c00c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57c00b4c10 0x7f57c00b5080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f57a0066320 0x7f57a00687d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 --2- 172.21.15.22:0/716577672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ac004870 0x7f57c00d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 >> 172.21.15.22:0/716577672 conn(0x7f57c0069960 msgr2=0x7f57c015dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:29.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 shutdown_connections 2023-06-23T18:04:29.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.454+0000 7f57c556c700 1 -- 172.21.15.22:0/716577672 wait complete. 2023-06-23T18:04:29.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-06-23T18:04:29.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 Processor -- start 2023-06-23T18:04:29.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 -- start start 2023-06-23T18:04:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb74015bf10 0x7fb74006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7400c9100 0x7fb7400ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74006c9b0 con 0x7fb74006c510 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74006cf70 con 0x7fb74015bf10 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73f59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74006cc90 con 0x7fb7400c9100 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73e59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb74015bf10 0x7fb74006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73e59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb74015bf10 0x7fb74006bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54366/0 (socket says 172.21.15.22:54366) 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73e59c700 1 -- 172.21.15.22:0/1820895213 learned_addr learned my addr 172.21.15.22:0/1820895213 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73ed9d700 1 --2- 172.21.15.22:0/1820895213 >> v2:172.21.15.169:3300/0 conn(0x7fb7400c9100 0x7fb7400ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73e59c700 1 -- 172.21.15.22:0/1820895213 >> v2:172.21.15.169:3300/0 conn(0x7fb7400c9100 msgr2=0x7fb7400ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:29.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.602+0000 7fb73e59c700 1 --2- 172.21.15.22:0/1820895213 >> v2:172.21.15.169:3300/0 conn(0x7fb7400c9100 0x7fb7400ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73e59c700 1 -- 172.21.15.22:0/1820895213 >> v1:172.21.15.169:6789/0 conn(0x7fb74006c510 legacy=0x7fb7400c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73e59c700 1 -- 172.21.15.22:0/1820895213 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400cc5e0 con 0x7fb74015bf10 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73e59c700 1 --2- 172.21.15.22:0/1820895213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74015bf10 0x7fb74006bfd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7400ad7f0 tx=0x7fb734009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd2f807f3e920bd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73d59a700 1 -- 172.21.15.22:0/1820895213 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011830 con 0x7fb74015bf10 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73d59a700 1 -- 172.21.15.22:0/1820895213 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734011990 con 0x7fb74015bf10 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73d59a700 1 -- 172.21.15.22:0/1820895213 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011d20 con 0x7fb74015bf10 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- 172.21.15.22:0/1820895213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74015bf10 msgr2=0x7fb74006bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1820895213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74015bf10 0x7fb74006bfd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7400ad7f0 tx=0x7fb734009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- 172.21.15.22:0/1820895213 shutdown_connections 2023-06-23T18:04:29.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1820895213 >> v2:172.21.15.169:3300/0 conn(0x7fb7400c9100 0x7fb7400ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1820895213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74015bf10 0x7fb74006bfd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- 172.21.15.22:0/1820895213 >> 172.21.15.22:0/1820895213 conn(0x7fb74006a5f0 msgr2=0x7fb7400caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:29.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- 172.21.15.22:0/1820895213 shutdown_connections 2023-06-23T18:04:29.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- 172.21.15.22:0/1820895213 wait complete. 2023-06-23T18:04:29.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 Processor -- start 2023-06-23T18:04:29.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.606+0000 7fb73f59e700 1 -- start start 2023-06-23T18:04:29.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb74006c510 0x7fb740159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb74015bf10 0x7fb7400b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b2d90 con 0x7fb7400c9100 2023-06-23T18:04:29.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b3070 con 0x7fb74006c510 2023-06-23T18:04:29.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b3350 con 0x7fb74015bf10 2023-06-23T18:04:29.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54368/0 (socket says 172.21.15.22:54368) 2023-06-23T18:04:29.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 -- 172.21.15.22:0/1172967252 learned_addr learned my addr 172.21.15.22:0/1172967252 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73ed9d700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb74015bf10 0x7fb7400b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73e59c700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb74006c510 0x7fb740159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 -- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb74015bf10 msgr2=0x7fb7400b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb74015bf10 0x7fb7400b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 -- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb74006c510 msgr2=0x7fb740159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb74006c510 0x7fb740159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 -- 172.21.15.22:0/1172967252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009570 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73dd9b700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c4d0 tx=0x7fb73000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300052d0 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730005430 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7400b3630 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300055a0 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400c7e50 con 0x7fb7400c9100 2023-06-23T18:04:29.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.610+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74004e4c0 con 0x7fb7400c9100 2023-06-23T18:04:29.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.614+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb7300038c0 con 0x7fb7400c9100 2023-06-23T18:04:29.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.614+0000 7fb72effd700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 0x7fb7200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:29.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.614+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb730083340 con 0x7fb7400c9100 2023-06-23T18:04:29.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.614+0000 7fb73e59c700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 0x7fb7200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:29.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.614+0000 7fb73e59c700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 0x7fb7200687a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:29.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.618+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb7300504a0 con 0x7fb7400c9100 2023-06-23T18:04:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.790+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7400b3ff0 con 0x7fb7400c9100 2023-06-23T18:04:29.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.794+0000 7fb72effd700 1 -- 172.21.15.22:0/1172967252 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb730053ac0 con 0x7fb7400c9100 2023-06-23T18:04:29.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 msgr2=0x7fb7200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 0x7fb7200687a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734008030 comp rx=0 tx=0).stop 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 msgr2=0x7fb7400c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c4d0 tx=0x7fb73000c7e0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 shutdown_connections 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb74015bf10 0x7fb7400b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7400c9100 0x7fb7400c7910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb7200662f0 0x7fb7200687a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 --2- 172.21.15.22:0/1172967252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb74006c510 0x7fb740159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 >> 172.21.15.22:0/1172967252 conn(0x7fb74006a5f0 msgr2=0x7fb7400ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 shutdown_connections 2023-06-23T18:04:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.798+0000 7fb73f59e700 1 -- 172.21.15.22:0/1172967252 wait complete. 2023-06-23T18:04:29.818 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-23T18:04:29.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:04:29.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-23T18:04:29.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-23T18:04:29.856 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:29.857 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:29.884 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:29.885 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:29.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.998+0000 7f5acac1b700 1 Processor -- start 2023-06-23T18:04:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:29.998+0000 7f5acac1b700 1 -- start start 2023-06-23T18:04:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5acac1b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5ac41580e0 0x7f5ac40cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5acac1b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5ac40713f0 0x7f5ac4158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5acac1b700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac40cc270 con 0x7f5ac40cd310 2023-06-23T18:04:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5acac1b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac40cc550 con 0x7f5ac40713f0 2023-06-23T18:04:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5acac1b700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac40cc830 con 0x7f5ac41580e0 2023-06-23T18:04:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5ac40713f0 0x7f5ac4158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5ac40713f0 0x7f5ac4158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54378/0 (socket says 172.21.15.22:54378) 2023-06-23T18:04:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 -- 172.21.15.22:0/3984751214 learned_addr learned my addr 172.21.15.22:0/3984751214 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5ac9c19700 1 --2- 172.21.15.22:0/3984751214 >> v2:172.21.15.169:3300/0 conn(0x7f5ac41580e0 0x7f5ac40cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 -- 172.21.15.22:0/3984751214 >> v2:172.21.15.169:3300/0 conn(0x7f5ac41580e0 msgr2=0x7f5ac40cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 --2- 172.21.15.22:0/3984751214 >> v2:172.21.15.169:3300/0 conn(0x7f5ac41580e0 0x7f5ac40cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 -- 172.21.15.22:0/3984751214 >> v1:172.21.15.169:6789/0 conn(0x7f5ac40cd310 legacy=0x7f5ac41584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 -- 172.21.15.22:0/3984751214 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40ccb10 con 0x7f5ac40713f0 2023-06-23T18:04:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5aca41a700 1 --2- 172.21.15.22:0/3984751214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac40713f0 0x7f5ac4158bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4071b10 tx=0x7f5abc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d0dd6099aa111a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5ac8c17700 1 -- 172.21.15.22:0/3984751214 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00b910 con 0x7f5ac40713f0 2023-06-23T18:04:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5ac8c17700 1 -- 172.21.15.22:0/3984751214 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc00ba70 con 0x7f5ac40713f0 2023-06-23T18:04:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.002+0000 7f5ac8c17700 1 -- 172.21.15.22:0/3984751214 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00be20 con 0x7f5ac40713f0 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- 172.21.15.22:0/3984751214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac40713f0 msgr2=0x7f5ac4158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3984751214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac40713f0 0x7f5ac4158bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4071b10 tx=0x7f5abc00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- 172.21.15.22:0/3984751214 shutdown_connections 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3984751214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac40713f0 0x7f5ac4158bc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3984751214 >> v2:172.21.15.169:3300/0 conn(0x7f5ac41580e0 0x7f5ac40cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- 172.21.15.22:0/3984751214 >> 172.21.15.22:0/3984751214 conn(0x7f5ac406a5f0 msgr2=0x7f5ac40caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- 172.21.15.22:0/3984751214 shutdown_connections 2023-06-23T18:04:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- 172.21.15.22:0/3984751214 wait complete. 2023-06-23T18:04:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 Processor -- start 2023-06-23T18:04:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- start start 2023-06-23T18:04:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ac40713f0 0x7f5ac40c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 0x7f5ac40c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 0x7f5ac4155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41501b0 con 0x7f5ac41580e0 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4154860 con 0x7f5ac40cd310 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5acac1b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4154b40 con 0x7f5ac40713f0 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.006+0000 7f5aca41a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 0x7f5ac4155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aca41a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 0x7f5ac4155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54382/0 (socket says 172.21.15.22:54382) 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aca41a700 1 -- 172.21.15.22:0/3855655765 learned_addr learned my addr 172.21.15.22:0/3855655765 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9c19700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ac40713f0 0x7f5ac40c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 0x7f5ac40c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 -- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ac40713f0 msgr2=0x7f5ac40c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ac40713f0 0x7f5ac40c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 -- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 msgr2=0x7f5ac4155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 0x7f5ac4155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 -- 172.21.15.22:0/3855655765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009b30 con 0x7f5ac40cd310 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5ac9418700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 0x7f5ac40c19e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c480 tx=0x7f5ac000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00052d0 con 0x7f5ac40cd310 2023-06-23T18:04:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0005430 con 0x7f5ac40cd310 2023-06-23T18:04:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac4154e20 con 0x7f5ac40cd310 2023-06-23T18:04:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00055a0 con 0x7f5ac40cd310 2023-06-23T18:04:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac41553e0 con 0x7f5ac40cd310 2023-06-23T18:04:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac404e4c0 con 0x7f5ac40cd310 2023-06-23T18:04:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5ac00038e0 con 0x7f5ac40cd310 2023-06-23T18:04:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.010+0000 7f5aba7fc700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 0x7f5aa80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.014+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5ac00837d0 con 0x7f5ac40cd310 2023-06-23T18:04:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.014+0000 7f5ac9c19700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 0x7f5aa80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.014+0000 7f5ac9c19700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 0x7f5aa80687a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ac415ba80 tx=0x7f5ab4009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.022+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5ac0050920 con 0x7f5ac40cd310 2023-06-23T18:04:30.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.186+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ac40b05b0 con 0x7f5ac40cd310 2023-06-23T18:04:30.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5aba7fc700 1 -- 172.21.15.22:0/3855655765 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f5ac0053f40 con 0x7f5ac40cd310 2023-06-23T18:04:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:30.199 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 msgr2=0x7f5aa80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 0x7f5aa80687a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ac415ba80 tx=0x7f5ab4009240 comp rx=0 tx=0).stop 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 msgr2=0x7f5ac40c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 0x7f5ac40c19e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c480 tx=0x7f5ac000c790 comp rx=0 tx=0).stop 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 shutdown_connections 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ac41580e0 0x7f5ac4155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f5aa80662f0 0x7f5aa80687a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac40cd310 0x7f5ac40c19e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 --2- 172.21.15.22:0/3855655765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ac40713f0 0x7f5ac40c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 >> 172.21.15.22:0/3855655765 conn(0x7f5ac406a5f0 msgr2=0x7f5ac41584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 shutdown_connections 2023-06-23T18:04:30.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.190+0000 7f5acac1b700 1 -- 172.21.15.22:0/3855655765 wait complete. 2023-06-23T18:04:30.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:30.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 Processor -- start 2023-06-23T18:04:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 -- start start 2023-06-23T18:04:30.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 0x7fd1880cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880ccb30 con 0x7fd1880d0730 2023-06-23T18:04:30.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cfcb0 con 0x7fd1880d1f30 2023-06-23T18:04:30.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.330+0000 7fd18de0c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880d2880 con 0x7fd1880cf3a0 2023-06-23T18:04:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18d60b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd1880d0730 0x7fd1880ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58336/0 (socket says 172.21.15.22:58336) 2023-06-23T18:04:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18d60b700 1 -- 172.21.15.22:0/3463614709 learned_addr learned my addr 172.21.15.22:0/3463614709 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18ce0a700 1 --2- 172.21.15.22:0/3463614709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 0x7fd1880cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834729767 0 0) 0x7fd1880ccb30 con 0x7fd1880d0730 2023-06-23T18:04:30.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1740035d0 con 0x7fd1880d0730 2023-06-23T18:04:30.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984221980 0 0) 0x7fd1880cfcb0 con 0x7fd1880d1f30 2023-06-23T18:04:30.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1880ccb30 con 0x7fd1880d1f30 2023-06-23T18:04:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18ce0a700 1 -- 172.21.15.22:0/3463614709 >> v1:172.21.15.169:6789/0 conn(0x7fd1880d1f30 legacy=0x7fd1880d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18ce0a700 1 -- 172.21.15.22:0/3463614709 >> v1:172.21.15.22:6789/0 conn(0x7fd1880d0730 legacy=0x7fd1880ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18ce0a700 1 -- 172.21.15.22:0/3463614709 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880d2b60 con 0x7fd1880cf3a0 2023-06-23T18:04:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18ce0a700 1 --2- 172.21.15.22:0/3463614709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 0x7fd1880cf770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd170009870 tx=0x7fd170009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02085e28ded636b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170005e50 con 0x7fd1880cf3a0 2023-06-23T18:04:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170006030 con 0x7fd1880cf3a0 2023-06-23T18:04:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd1877fe700 1 -- 172.21.15.22:0/3463614709 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700063c0 con 0x7fd1880cf3a0 2023-06-23T18:04:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 -- 172.21.15.22:0/3463614709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 msgr2=0x7fd1880cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 --2- 172.21.15.22:0/3463614709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 0x7fd1880cf770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd170009870 tx=0x7fd170009b80 comp rx=0 tx=0).stop 2023-06-23T18:04:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 -- 172.21.15.22:0/3463614709 shutdown_connections 2023-06-23T18:04:30.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 --2- 172.21.15.22:0/3463614709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880cf3a0 0x7fd1880cf770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 -- 172.21.15.22:0/3463614709 >> 172.21.15.22:0/3463614709 conn(0x7fd18806a280 msgr2=0x7fd18809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 -- 172.21.15.22:0/3463614709 shutdown_connections 2023-06-23T18:04:30.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.334+0000 7fd18de0c700 1 -- 172.21.15.22:0/3463614709 wait complete. 2023-06-23T18:04:30.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 Processor -- start 2023-06-23T18:04:30.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- start start 2023-06-23T18:04:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 0x7fd18815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1880d1f30 0x7fd1880c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815c0e0 con 0x7fd1880cf3a0 2023-06-23T18:04:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b61b0 con 0x7fd1880d1f30 2023-06-23T18:04:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815c330 con 0x7fd1880d0730 2023-06-23T18:04:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54392/0 (socket says 172.21.15.22:54392) 2023-06-23T18:04:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 -- 172.21.15.22:0/1807628703 learned_addr learned my addr 172.21.15.22:0/1807628703 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18d60b700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1880d1f30 0x7fd1880c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd187fff700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 0x7fd18815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 -- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 msgr2=0x7fd18815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 0x7fd18815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 -- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1880d1f30 msgr2=0x7fd1880c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1880d1f30 0x7fd1880c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 -- 172.21.15.22:0/1807628703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009450 con 0x7fd1880cf3a0 2023-06-23T18:04:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd187fff700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 0x7fd18815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18ce0a700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd170013040 tx=0x7fd170004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170006970 con 0x7fd1880cf3a0 2023-06-23T18:04:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170020070 con 0x7fd1880cf3a0 2023-06-23T18:04:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18815c610 con 0x7fd1880cf3a0 2023-06-23T18:04:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170025670 con 0x7fd1880cf3a0 2023-06-23T18:04:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.338+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18815e850 con 0x7fd1880cf3a0 2023-06-23T18:04:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18815cb30 con 0x7fd1880cf3a0 2023-06-23T18:04:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd170022030 con 0x7fd1880cf3a0 2023-06-23T18:04:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd1857fa700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 0x7fd1680687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd170085620 con 0x7fd1880cf3a0 2023-06-23T18:04:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd187fff700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 0x7fd1680687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.342+0000 7fd187fff700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 0x7fd1680687a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd18806d280 tx=0x7fd178009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.350+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd1700528a0 con 0x7fd1880cf3a0 2023-06-23T18:04:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.514+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd18815e9e0 con 0x7fd1880cf3a0 2023-06-23T18:04:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.514+0000 7fd1857fa700 1 -- 172.21.15.22:0/1807628703 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd170055ec0 con 0x7fd1880cf3a0 2023-06-23T18:04:30.523 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:30.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 msgr2=0x7fd1680687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 0x7fd1680687a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd18806d280 tx=0x7fd178009040 comp rx=0 tx=0).stop 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 msgr2=0x7fd188153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd170013040 tx=0x7fd170004770 comp rx=0 tx=0).stop 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 shutdown_connections 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1680662f0 0x7fd1680687a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1880d1f30 0x7fd1880c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1880d0730 0x7fd18815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 --2- 172.21.15.22:0/1807628703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1880cf3a0 0x7fd188153b30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 >> 172.21.15.22:0/1807628703 conn(0x7fd18806a280 msgr2=0x7fd1880b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 shutdown_connections 2023-06-23T18:04:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.518+0000 7fd18de0c700 1 -- 172.21.15.22:0/1807628703 wait complete. 2023-06-23T18:04:30.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:30.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 Processor -- start 2023-06-23T18:04:30.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- start start 2023-06-23T18:04:30.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe3d8160890 0x7fe3d8160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d8161550 con 0x7fe3d80706c0 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d806fbe0 con 0x7fe3d80b1140 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d8161830 con 0x7fe3d8160890 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe3d8160890 0x7fe3d8160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe3d80706c0 0x7fe3d8070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58350/0 (socket says 172.21.15.22:58350) 2023-06-23T18:04:30.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 -- 172.21.15.22:0/4183183391 learned_addr learned my addr 172.21.15.22:0/4183183391 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 -- 172.21.15.22:0/4183183391 >> v1:172.21.15.169:6789/0 conn(0x7fe3d80b1140 legacy=0x7fe3d80b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 -- 172.21.15.22:0/4183183391 >> v1:172.21.15.22:6789/0 conn(0x7fe3d80706c0 legacy=0x7fe3d8070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d77fe700 1 -- 172.21.15.22:0/4183183391 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768592992 0 0) 0x7fe3d8161550 con 0x7fe3d80706c0 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 -- 172.21.15.22:0/4183183391 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d806fec0 con 0x7fe3d8160890 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 --2- 172.21.15.22:0/4183183391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d8160ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009850 tx=0x7fe3c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e950d4f9e62b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d77fe700 1 -- 172.21.15.22:0/4183183391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011830 con 0x7fe3d8160890 2023-06-23T18:04:30.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d77fe700 1 -- 172.21.15.22:0/4183183391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4011990 con 0x7fe3d8160890 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d77fe700 1 -- 172.21.15.22:0/4183183391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011d20 con 0x7fe3d8160890 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4183183391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 msgr2=0x7fe3d8160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4183183391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d8160ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009850 tx=0x7fe3c4009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4183183391 shutdown_connections 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4183183391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d8160ce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4183183391 >> 172.21.15.22:0/4183183391 conn(0x7fe3d8060560 msgr2=0x7fe3d806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4183183391 shutdown_connections 2023-06-23T18:04:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4183183391 wait complete. 2023-06-23T18:04:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 Processor -- start 2023-06-23T18:04:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- start start 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d80706c0 0x7fe3d80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d80b1140 0x7fe3d8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d81f7030 con 0x7fe3d8160890 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d8154190 con 0x7fe3d80b1140 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d81530f0 con 0x7fe3d80706c0 2023-06-23T18:04:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54422/0 (socket says 172.21.15.22:54422) 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 -- 172.21.15.22:0/4155478291 learned_addr learned my addr 172.21.15.22:0/4155478291 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dcd94700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d80706c0 0x7fe3d80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d7fff700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d80b1140 0x7fe3d8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 -- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d80706c0 msgr2=0x7fe3d80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d80706c0 0x7fe3d80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 -- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d80b1140 msgr2=0x7fe3d8152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d80b1140 0x7fe3d8152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 -- 172.21.15.22:0/4155478291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c40094b0 con 0x7fe3d8160890 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3dd595700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000cba0 tx=0x7fe3d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000ab80 con 0x7fe3d8160890 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d000ad00 con 0x7fe3d8160890 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d81f6c70 con 0x7fe3d8160890 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0007310 con 0x7fe3d8160890 2023-06-23T18:04:30.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.650+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d81525b0 con 0x7fe3d8160890 2023-06-23T18:04:30.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d81f7170 con 0x7fe3d8160890 2023-06-23T18:04:30.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe3d0007470 con 0x7fe3d8160890 2023-06-23T18:04:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3d57fa700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 0x7fe3bc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe3d0083c70 con 0x7fe3d8160890 2023-06-23T18:04:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3dcd94700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 0x7fe3bc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.654+0000 7fe3dcd94700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 0x7fe3bc0687a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000f60 tx=0x7fe3cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.662+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe3d81f7170 con 0x7fe3d8160890 2023-06-23T18:04:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3d81f7170 con 0x7fe3d8160890 2023-06-23T18:04:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3d57fa700 1 -- 172.21.15.22:0/4155478291 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3d81f7170 con 0x7fe3d8160890 2023-06-23T18:04:30.840 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 msgr2=0x7fe3bc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 0x7fe3bc0687a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000f60 tx=0x7fe3cc009040 comp rx=0 tx=0).stop 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 msgr2=0x7fe3d815b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.826+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000cba0 tx=0x7fe3d000cf60 comp rx=0 tx=0).stop 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 shutdown_connections 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d8160890 0x7fe3d815b340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe3bc0662f0 0x7fe3bc0687a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d80b1140 0x7fe3d8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 --2- 172.21.15.22:0/4155478291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d80706c0 0x7fe3d80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 >> 172.21.15.22:0/4155478291 conn(0x7fe3d8060560 msgr2=0x7fe3d80b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 shutdown_connections 2023-06-23T18:04:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.830+0000 7fe3deff8700 1 -- 172.21.15.22:0/4155478291 wait complete. 2023-06-23T18:04:30.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 Processor -- start 2023-06-23T18:04:30.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 -- start start 2023-06-23T18:04:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe228159c30 0x7fe22815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2280cd8e0 0x7fe2280cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2280b1d60 0x7fe2280b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280b2720 con 0x7fe2280b1d60 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22815a700 con 0x7fe2280cd8e0 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22edb0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280ce7b0 con 0x7fe228159c30 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2280b1d60 0x7fe2280b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2280b1d60 0x7fe2280b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54430/0 (socket says 172.21.15.22:54430) 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 -- 172.21.15.22:0/664953182 learned_addr learned my addr 172.21.15.22:0/664953182 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22ddae700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe228159c30 0x7fe22815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22d5ad700 1 --2- 172.21.15.22:0/664953182 >> v2:172.21.15.169:3300/0 conn(0x7fe2280cd8e0 0x7fe2280cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 -- 172.21.15.22:0/664953182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe228159c30 msgr2=0x7fe22815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe228159c30 0x7fe22815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 -- 172.21.15.22:0/664953182 >> v2:172.21.15.169:3300/0 conn(0x7fe2280cd8e0 msgr2=0x7fe2280cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 --2- 172.21.15.22:0/664953182 >> v2:172.21.15.169:3300/0 conn(0x7fe2280cd8e0 0x7fe2280cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 -- 172.21.15.22:0/664953182 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280cea90 con 0x7fe2280b1d60 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22e5af700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2280b1d60 0x7fe2280b21e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe2280d1370 tx=0x7fe22000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9c56f425b153b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22cdac700 1 -- 172.21.15.22:0/664953182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000e070 con 0x7fe2280b1d60 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22cdac700 1 -- 172.21.15.22:0/664953182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000b910 con 0x7fe2280b1d60 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.966+0000 7fe22cdac700 1 -- 172.21.15.22:0/664953182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000bcc0 con 0x7fe2280b1d60 2023-06-23T18:04:30.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/664953182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2280b1d60 msgr2=0x7fe2280b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2280b1d60 0x7fe2280b21e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe2280d1370 tx=0x7fe22000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/664953182 shutdown_connections 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2280b1d60 0x7fe2280b21e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- 172.21.15.22:0/664953182 >> v2:172.21.15.169:3300/0 conn(0x7fe2280cd8e0 0x7fe2280cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- 172.21.15.22:0/664953182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe228159c30 0x7fe22815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/664953182 >> 172.21.15.22:0/664953182 conn(0x7fe2280ccd60 msgr2=0x7fe2280cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/664953182 shutdown_connections 2023-06-23T18:04:30.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/664953182 wait complete. 2023-06-23T18:04:30.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 Processor -- start 2023-06-23T18:04:30.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- start start 2023-06-23T18:04:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280b1d60 0x7fe2280b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2280cd8e0 0x7fe2281f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280af070 con 0x7fe228159c30 2023-06-23T18:04:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228155660 con 0x7fe2280b1d60 2023-06-23T18:04:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe228155940 con 0x7fe2280cd8e0 2023-06-23T18:04:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54446/0 (socket says 172.21.15.22:54446) 2023-06-23T18:04:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 -- 172.21.15.22:0/3857075277 learned_addr learned my addr 172.21.15.22:0/3857075277 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22ddae700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280b1d60 0x7fe2280b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22d5ad700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2280cd8e0 0x7fe2281f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 -- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2280cd8e0 msgr2=0x7fe2281f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2280cd8e0 0x7fe2281f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 -- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280b1d60 msgr2=0x7fe2280b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280b1d60 0x7fe2280b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 -- 172.21.15.22:0/3857075277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009b30 con 0x7fe228159c30 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22e5af700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe228072e10 tx=0x7fe220004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200099b0 con 0x7fe228159c30 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000ce80 con 0x7fe228159c30 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22806ab10 con 0x7fe228159c30 2023-06-23T18:04:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220004490 con 0x7fe228159c30 2023-06-23T18:04:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.970+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe228155c20 con 0x7fe228159c30 2023-06-23T18:04:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22806b030 con 0x7fe228159c30 2023-06-23T18:04:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe2200045f0 con 0x7fe228159c30 2023-06-23T18:04:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe21e7fc700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 0x7fe20c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe2200853b0 con 0x7fe228159c30 2023-06-23T18:04:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe22ddae700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 0x7fe20c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.974+0000 7fe22ddae700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 0x7fe20c0687a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe224009b50 tx=0x7fe224009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:30.982+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe2200525c0 con 0x7fe228159c30 2023-06-23T18:04:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.142+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe22804e4c0 con 0x7fe228159c30 2023-06-23T18:04:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.142+0000 7fe21e7fc700 1 -- 172.21.15.22:0/3857075277 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2200048a0 con 0x7fe228159c30 2023-06-23T18:04:31.157 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:31.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 msgr2=0x7fe20c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 0x7fe20c0687a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe224009b50 tx=0x7fe224009240 comp rx=0 tx=0).stop 2023-06-23T18:04:31.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 msgr2=0x7fe2280b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe228072e10 tx=0x7fe220004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 shutdown_connections 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe228159c30 0x7fe2280b3dc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2280cd8e0 0x7fe2281f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe20c0662f0 0x7fe20c0687a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 --2- 172.21.15.22:0/3857075277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280b1d60 0x7fe2280b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 >> 172.21.15.22:0/3857075277 conn(0x7fe2280ccd60 msgr2=0x7fe2280cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 shutdown_connections 2023-06-23T18:04:31.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.146+0000 7fe22edb0700 1 -- 172.21.15.22:0/3857075277 wait complete. 2023-06-23T18:04:31.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:31.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 Processor -- start 2023-06-23T18:04:31.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 -- start start 2023-06-23T18:04:31.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7a200d1f30 0x7f7a200d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a200ccb30 con 0x7f7a200cf3a0 2023-06-23T18:04:31.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a200d2880 con 0x7f7a200d2360 2023-06-23T18:04:31.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200d2b60 con 0x7f7a200d1f30 2023-06-23T18:04:31.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1e7fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7a200d1f30 0x7f7a200d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1effd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7a200cf3a0 0x7f7a200cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58358/0 (socket says 172.21.15.22:58358) 2023-06-23T18:04:31.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1effd700 1 -- 172.21.15.22:0/1265244001 learned_addr learned my addr 172.21.15.22:0/1265244001 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:31.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1dffb700 1 -- 172.21.15.22:0/1265244001 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337982496 0 0) 0x7f7a200ccb30 con 0x7f7a200cf3a0 2023-06-23T18:04:31.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1dffb700 1 -- 172.21.15.22:0/1265244001 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a0c0035d0 con 0x7f7a200cf3a0 2023-06-23T18:04:31.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1e7fc700 1 -- 172.21.15.22:0/1265244001 >> v1:172.21.15.169:6789/0 conn(0x7f7a200d2360 legacy=0x7f7a200ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1e7fc700 1 -- 172.21.15.22:0/1265244001 >> v1:172.21.15.22:6789/0 conn(0x7f7a200cf3a0 legacy=0x7f7a200cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.286+0000 7f7a1e7fc700 1 -- 172.21.15.22:0/1265244001 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200d2e40 con 0x7f7a200d1f30 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1e7fc700 1 --2- 172.21.15.22:0/1265244001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d1f30 0x7f7a200d0020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009850 tx=0x7f7a08009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1d196b4e6cc1af server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1dffb700 1 -- 172.21.15.22:0/1265244001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08011830 con 0x7f7a200d1f30 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1dffb700 1 -- 172.21.15.22:0/1265244001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08011990 con 0x7f7a200d1f30 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1dffb700 1 -- 172.21.15.22:0/1265244001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08011d40 con 0x7f7a200d1f30 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- 172.21.15.22:0/1265244001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d1f30 msgr2=0x7f7a200d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/1265244001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d1f30 0x7f7a200d0020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009850 tx=0x7f7a08009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:31.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- 172.21.15.22:0/1265244001 shutdown_connections 2023-06-23T18:04:31.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/1265244001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d1f30 0x7f7a200d0020 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- 172.21.15.22:0/1265244001 >> 172.21.15.22:0/1265244001 conn(0x7f7a2006a280 msgr2=0x7f7a2009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- 172.21.15.22:0/1265244001 shutdown_connections 2023-06-23T18:04:31.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- 172.21.15.22:0/1265244001 wait complete. 2023-06-23T18:04:31.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 Processor -- start 2023-06-23T18:04:31.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.290+0000 7f7a1ffff700 1 -- start start 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a200cf3a0 0x7f7a201573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a200d1f30 0x7f7a201f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2015ede0 con 0x7f7a200d2360 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2015d300 con 0x7f7a200d1f30 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2015caf0 con 0x7f7a200cf3a0 2023-06-23T18:04:31.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a200cf3a0 0x7f7a201573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54464/0 (socket says 172.21.15.22:54464) 2023-06-23T18:04:31.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 -- 172.21.15.22:0/971657823 learned_addr learned my addr 172.21.15.22:0/971657823 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:31.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1e7fc700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a200d1f30 0x7f7a201f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 -- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a200cf3a0 msgr2=0x7f7a201573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a200cf3a0 0x7f7a201573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 -- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a200d1f30 msgr2=0x7f7a201f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a200d1f30 0x7f7a201f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 -- 172.21.15.22:0/971657823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080094b0 con 0x7f7a200d2360 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1f7fe700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800cb50 tx=0x7f7a1800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003c20 con 0x7f7a200d2360 2023-06-23T18:04:31.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18003d80 con 0x7f7a200d2360 2023-06-23T18:04:31.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a20157930 con 0x7f7a200d2360 2023-06-23T18:04:31.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800aa20 con 0x7f7a200d2360 2023-06-23T18:04:31.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2015d020 con 0x7f7a200d2360 2023-06-23T18:04:31.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.294+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2015ef20 con 0x7f7a200d2360 2023-06-23T18:04:31.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.298+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7a1800ab80 con 0x7f7a200d2360 2023-06-23T18:04:31.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.298+0000 7f7a17fff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 0x7f7a000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.298+0000 7f7a1effd700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 0x7f7a000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.298+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7a18083fe0 con 0x7f7a200d2360 2023-06-23T18:04:31.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.298+0000 7f7a1effd700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 0x7f7a000687a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a100010e0 tx=0x7f7a10007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.306+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7a2015ef20 con 0x7f7a200d2360 2023-06-23T18:04:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.466+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a2015ef20 con 0x7f7a200d2360 2023-06-23T18:04:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a17fff700 1 -- 172.21.15.22:0/971657823 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a2015ef20 con 0x7f7a200d2360 2023-06-23T18:04:31.482 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 msgr2=0x7f7a000687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 0x7f7a000687a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a100010e0 tx=0x7f7a10007c90 comp rx=0 tx=0).stop 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 msgr2=0x7f7a200b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800cb50 tx=0x7f7a1800ce60 comp rx=0 tx=0).stop 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 shutdown_connections 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a200d2360 0x7f7a200b0280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7a000662f0 0x7f7a000687a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a200d1f30 0x7f7a201f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 --2- 172.21.15.22:0/971657823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a200cf3a0 0x7f7a201573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 >> 172.21.15.22:0/971657823 conn(0x7f7a2006a280 msgr2=0x7f7a200b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 shutdown_connections 2023-06-23T18:04:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.470+0000 7f7a1ffff700 1 -- 172.21.15.22:0/971657823 wait complete. 2023-06-23T18:04:31.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.606+0000 7fb928754700 1 Processor -- start 2023-06-23T18:04:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.606+0000 7fb928754700 1 -- start start 2023-06-23T18:04:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9200b1a00 0x7fb920166a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9200d0bd0 con 0x7fb9200b7dc0 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9201606d0 con 0x7fb9200cd120 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200b8190 con 0x7fb9200b1a00 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9264f0700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb9200b7dc0 0x7fb92015f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58362/0 (socket says 172.21.15.22:58362) 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9264f0700 1 -- 172.21.15.22:0/3372261496 learned_addr learned my addr 172.21.15.22:0/3372261496 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb926cf1700 1 --2- 172.21.15.22:0/3372261496 >> v2:172.21.15.169:3300/0 conn(0x7fb9200b1a00 0x7fb920166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9254ee700 1 -- 172.21.15.22:0/3372261496 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143405985 0 0) 0x7fb9200d0bd0 con 0x7fb9200b7dc0 2023-06-23T18:04:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9254ee700 1 -- 172.21.15.22:0/3372261496 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb90c0035d0 con 0x7fb9200b7dc0 2023-06-23T18:04:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb926cf1700 1 -- 172.21.15.22:0/3372261496 >> v1:172.21.15.169:6789/0 conn(0x7fb9200cd120 legacy=0x7fb92015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb926cf1700 1 -- 172.21.15.22:0/3372261496 >> v1:172.21.15.22:6789/0 conn(0x7fb9200b7dc0 legacy=0x7fb92015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb926cf1700 1 -- 172.21.15.22:0/3372261496 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200cf850 con 0x7fb9200b1a00 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb926cf1700 1 --2- 172.21.15.22:0/3372261496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb920166a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb918009d90 tx=0x7fb91800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f13824657501a2f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9254ee700 1 -- 172.21.15.22:0/3372261496 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800e070 con 0x7fb9200b1a00 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9254ee700 1 -- 172.21.15.22:0/3372261496 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91800bb20 con 0x7fb9200b1a00 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb9254ee700 1 -- 172.21.15.22:0/3372261496 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800be90 con 0x7fb9200b1a00 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- 172.21.15.22:0/3372261496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 msgr2=0x7fb920166a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 --2- 172.21.15.22:0/3372261496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb920166a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb918009d90 tx=0x7fb91800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- 172.21.15.22:0/3372261496 shutdown_connections 2023-06-23T18:04:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 --2- 172.21.15.22:0/3372261496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb920166a30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- 172.21.15.22:0/3372261496 >> 172.21.15.22:0/3372261496 conn(0x7fb92006b3e0 msgr2=0x7fb92006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- 172.21.15.22:0/3372261496 shutdown_connections 2023-06-23T18:04:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.610+0000 7fb928754700 1 -- 172.21.15.22:0/3372261496 wait complete. 2023-06-23T18:04:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 Processor -- start 2023-06-23T18:04:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- start start 2023-06-23T18:04:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb92015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9200b7dc0 0x7fb9201f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200caf10 con 0x7fb9200cd120 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200ca090 con 0x7fb9200b1a00 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200ca290 con 0x7fb9200b7dc0 2023-06-23T18:04:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54474/0 (socket says 172.21.15.22:54474) 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 -- 172.21.15.22:0/2407972555 learned_addr learned my addr 172.21.15.22:0/2407972555 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb9264f0700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb92015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb925cef700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9200b7dc0 0x7fb9201f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 -- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9200b7dc0 msgr2=0x7fb9201f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9200b7dc0 0x7fb9201f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 -- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 msgr2=0x7fb92015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb92015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 -- 172.21.15.22:0/2407972555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009a70 con 0x7fb9200cd120 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb926cf1700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b370 tx=0x7fb918004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180048b0 con 0x7fb9200cd120 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918020070 con 0x7fb9200cd120 2023-06-23T18:04:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9200ca570 con 0x7fb9200cd120 2023-06-23T18:04:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180276c0 con 0x7fb9200cd120 2023-06-23T18:04:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.614+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9200cab30 con 0x7fb9200cd120 2023-06-23T18:04:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9200cb050 con 0x7fb9200cd120 2023-06-23T18:04:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb918012430 con 0x7fb9200cd120 2023-06-23T18:04:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb912ffd700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 0x7fb9040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb9264f0700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 0x7fb9040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb918022070 con 0x7fb9200cd120 2023-06-23T18:04:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.618+0000 7fb9264f0700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 0x7fb9040687a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb91c007fd0 tx=0x7fb91c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.626+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb9200cb050 con 0x7fb9200cd120 2023-06-23T18:04:31.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.786+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb9200cb050 con 0x7fb9200cd120 2023-06-23T18:04:31.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.786+0000 7fb912ffd700 1 -- 172.21.15.22:0/2407972555 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb9200cb050 con 0x7fb9200cd120 2023-06-23T18:04:31.800 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:31.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 msgr2=0x7fb9040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 0x7fb9040687a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb91c007fd0 tx=0x7fb91c007de0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 msgr2=0x7fb92015f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b370 tx=0x7fb918004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 shutdown_connections 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9200cd120 0x7fb92015f5d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9200b7dc0 0x7fb9201f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb9040662f0 0x7fb9040687a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 --2- 172.21.15.22:0/2407972555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9200b1a00 0x7fb92015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 >> 172.21.15.22:0/2407972555 conn(0x7fb92006b3e0 msgr2=0x7fb9200b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 shutdown_connections 2023-06-23T18:04:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.790+0000 7fb928754700 1 -- 172.21.15.22:0/2407972555 wait complete. 2023-06-23T18:04:31.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 Processor -- start 2023-06-23T18:04:31.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 -- start start 2023-06-23T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 0x7f0d6c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c0b1840 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c15a0f0 con 0x7f0d6c0b4ec0 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.918+0000 7f0d73e96700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c06a6f0 con 0x7f0d6c0cb810 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54478/0 (socket says 172.21.15.22:54478) 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 -- 172.21.15.22:0/3801274421 learned_addr learned my addr 172.21.15.22:0/3801274421 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d71431700 1 --2- 172.21.15.22:0/3801274421 >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 0x7f0d6c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d71c32700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 -- 172.21.15.22:0/3801274421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 msgr2=0x7f0d6c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 -- 172.21.15.22:0/3801274421 >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 msgr2=0x7f0d6c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 --2- 172.21.15.22:0/3801274421 >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 0x7f0d6c0cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 -- 172.21.15.22:0/3801274421 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0cc120 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d71431700 1 --2- 172.21.15.22:0/3801274421 >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 0x7f0d6c0cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d72433700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c06a9f0 tx=0x7f0d6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f448fd29f6a978da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d70c30700 1 -- 172.21.15.22:0/3801274421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400e070 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d70c30700 1 -- 172.21.15.22:0/3801274421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400b9c0 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d70c30700 1 -- 172.21.15.22:0/3801274421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400bd70 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 -- 172.21.15.22:0/3801274421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 msgr2=0x7f0d6c0b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c06a9f0 tx=0x7f0d6400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 -- 172.21.15.22:0/3801274421 shutdown_connections 2023-06-23T18:04:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c0b1300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 --2- 172.21.15.22:0/3801274421 >> v2:172.21.15.169:3300/0 conn(0x7f0d6c0b4ec0 0x7f0d6c0cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 --2- 172.21.15.22:0/3801274421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 -- 172.21.15.22:0/3801274421 >> 172.21.15.22:0/3801274421 conn(0x7f0d6c06da30 msgr2=0x7f0d6c06de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 -- 172.21.15.22:0/3801274421 shutdown_connections 2023-06-23T18:04:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.922+0000 7f0d73e96700 1 -- 172.21.15.22:0/3801274421 wait complete. 2023-06-23T18:04:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 Processor -- start 2023-06-23T18:04:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 -- start start 2023-06-23T18:04:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6c0b4ec0 0x7f0d6c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c15b330 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c148e90 con 0x7f0d6c0b4ec0 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d73e96700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c15a290 con 0x7f0d6c0cb810 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d71c32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d71c32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54486/0 (socket says 172.21.15.22:54486) 2023-06-23T18:04:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d71c32700 1 -- 172.21.15.22:0/2786067004 learned_addr learned my addr 172.21.15.22:0/2786067004 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d71431700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6c0b4ec0 0x7f0d6c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.926+0000 7f0d72433700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 -- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 msgr2=0x7f0d6c06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c06e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 -- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6c0b4ec0 msgr2=0x7f0d6c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6c0b4ec0 0x7f0d6c06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 -- 172.21.15.22:0/2786067004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64009b30 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d71c32700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e760 tx=0x7f0d6800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800c070 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d68013610 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68013980 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6c15a570 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c15ab30 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6c158bc0 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.930+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0d68013ae0 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.934+0000 7f0d627fc700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 0x7f0d500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.934+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0d68083df0 con 0x7f0d6c0b0e80 2023-06-23T18:04:31.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.934+0000 7f0d71431700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 0x7f0d500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:31.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.934+0000 7f0d71431700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 0x7f0d500687a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0b0e10 tx=0x7f0d5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:31.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:31.942+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f0d680549b0 con 0x7f0d6c0b0e80 2023-06-23T18:04:32.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.110+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d6c04dc70 con 0x7f0d6c0b0e80 2023-06-23T18:04:32.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.110+0000 7f0d627fc700 1 -- 172.21.15.22:0/2786067004 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0d68054540 con 0x7f0d6c0b0e80 2023-06-23T18:04:32.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 msgr2=0x7f0d500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 0x7f0d500687a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0b0e10 tx=0x7f0d5c008040 comp rx=0 tx=0).stop 2023-06-23T18:04:32.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 msgr2=0x7f0d6c148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e760 tx=0x7f0d6800ea70 comp rx=0 tx=0).stop 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 shutdown_connections 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6c0cb810 0x7f0d6c06e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0d500662f0 0x7f0d500687a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6c0b4ec0 0x7f0d6c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 --2- 172.21.15.22:0/2786067004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d6c0b0e80 0x7f0d6c148950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 >> 172.21.15.22:0/2786067004 conn(0x7f0d6c06da30 msgr2=0x7f0d6c0b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 shutdown_connections 2023-06-23T18:04:32.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.114+0000 7f0d73e96700 1 -- 172.21.15.22:0/2786067004 wait complete. 2023-06-23T18:04:32.130 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:32.140 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:32.141 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:32.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:32.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 Processor -- start 2023-06-23T18:04:32.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 -- start start 2023-06-23T18:04:32.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e04159c30 0x7f3e040afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e040cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e040b0420 con 0x7f3e040affd0 2023-06-23T18:04:32.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0406d560 con 0x7f3e04159c30 2023-06-23T18:04:32.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.250+0000 7f3e08cf0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0406d0d0 con 0x7f3e04071450 2023-06-23T18:04:32.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e040cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e02ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3e040affd0 0x7f3e04070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:37434/0 (socket says 172.21.15.22:37434) 2023-06-23T18:04:32.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e037fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e04159c30 0x7f3e040afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e02ffd700 1 -- 172.21.15.22:0/199581781 learned_addr learned my addr 172.21.15.22:0/199581781 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:32.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e027fc700 1 -- 172.21.15.22:0/199581781 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154059914 0 0) 0x7f3df4001110 con 0x7f3e040affd0 2023-06-23T18:04:32.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e027fc700 1 -- 172.21.15.22:0/199581781 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dec0035d0 con 0x7f3e040affd0 2023-06-23T18:04:32.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 -- 172.21.15.22:0/199581781 >> v2:172.21.15.169:3300/0 conn(0x7f3e04159c30 msgr2=0x7f3e040afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 --2- 172.21.15.22:0/199581781 >> v2:172.21.15.169:3300/0 conn(0x7f3e04159c30 0x7f3e040afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 -- 172.21.15.22:0/199581781 >> v1:172.21.15.169:6789/0 conn(0x7f3e040affd0 legacy=0x7f3e04070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 -- 172.21.15.22:0/199581781 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040718d0 con 0x7f3e04071450 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e03fff700 1 --2- 172.21.15.22:0/199581781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e040cbbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009d90 tx=0x7f3dfc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0062ac169de969 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e027fc700 1 -- 172.21.15.22:0/199581781 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00e070 con 0x7f3e04071450 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e027fc700 1 -- 172.21.15.22:0/199581781 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00bb20 con 0x7f3e04071450 2023-06-23T18:04:32.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e027fc700 1 -- 172.21.15.22:0/199581781 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00be90 con 0x7f3e04071450 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 -- 172.21.15.22:0/199581781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 msgr2=0x7f3e040cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/199581781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e040cbbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009d90 tx=0x7f3dfc00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 -- 172.21.15.22:0/199581781 shutdown_connections 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/199581781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e040cbbe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/199581781 >> v2:172.21.15.169:3300/0 conn(0x7f3e04159c30 0x7f3e040afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 -- 172.21.15.22:0/199581781 >> 172.21.15.22:0/199581781 conn(0x7f3e0406a5f0 msgr2=0x7f3e040caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 -- 172.21.15.22:0/199581781 shutdown_connections 2023-06-23T18:04:32.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.254+0000 7f3e08cf0700 1 -- 172.21.15.22:0/199581781 wait complete. 2023-06-23T18:04:32.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 Processor -- start 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- start start 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e041f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e04159c30 0x7f3e041f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041f0e60 con 0x7f3e040affd0 2023-06-23T18:04:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c95a0 con 0x7f3e04159c30 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c9880 con 0x7f3e04071450 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54498/0 (socket says 172.21.15.22:54498) 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 -- 172.21.15.22:0/3013202236 learned_addr learned my addr 172.21.15.22:0/3013202236 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e03fff700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e04159c30 0x7f3e041f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e037fe700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e041f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 -- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 msgr2=0x7f3e041f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e041f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 -- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e04159c30 msgr2=0x7f3e041f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e04159c30 0x7f3e041f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 -- 172.21.15.22:0/3013202236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009a70 con 0x7f3e040affd0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e02ffd700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c4f0 tx=0x7f3df400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4005d90 con 0x7f3e040affd0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df40070d0 con 0x7f3e040affd0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e040c9b60 con 0x7f3e040affd0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4007480 con 0x7f3e040affd0 2023-06-23T18:04:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.258+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e041f5280 con 0x7f3e040affd0 2023-06-23T18:04:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0404e4c0 con 0x7f3e040affd0 2023-06-23T18:04:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3df40075e0 con 0x7f3e040affd0 2023-06-23T18:04:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3debfff700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 0x7f3df00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3df4083380 con 0x7f3e040affd0 2023-06-23T18:04:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3e037fe700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 0x7f3df00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.262+0000 7f3e037fe700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 0x7f3df00687a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e040c8780 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.270+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3df40504d0 con 0x7f3e040affd0 2023-06-23T18:04:32.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.438+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e040b0420 con 0x7f3e040affd0 2023-06-23T18:04:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.438+0000 7f3debfff700 1 -- 172.21.15.22:0/3013202236 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f3e040b0420 con 0x7f3e040affd0 2023-06-23T18:04:32.453 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:32.453 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 msgr2=0x7f3df00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 0x7f3df00687a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e040c8780 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2023-06-23T18:04:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 msgr2=0x7f3e0415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c4f0 tx=0x7f3df400c800 comp rx=0 tx=0).stop 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 shutdown_connections 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3df00662f0 0x7f3df00687a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e04159c30 0x7f3e041f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e040affd0 0x7f3e0415cb60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 --2- 172.21.15.22:0/3013202236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e04071450 0x7f3e041f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 >> 172.21.15.22:0/3013202236 conn(0x7f3e0406a5f0 msgr2=0x7f3e040cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 shutdown_connections 2023-06-23T18:04:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.442+0000 7f3e08cf0700 1 -- 172.21.15.22:0/3013202236 wait complete. 2023-06-23T18:04:32.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:32.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 Processor -- start 2023-06-23T18:04:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 -- start start 2023-06-23T18:04:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff2c0cf3a0 0x7eff2c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff2c0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0ccb30 con 0x7eff2c0d1f30 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0cfcb0 con 0x7eff2c0cf3a0 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31c49700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0d2880 con 0x7eff2c0d0730 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7eff2c0d1f30 0x7eff2c0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58366/0 (socket says 172.21.15.22:58366) 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2bfff700 1 -- 172.21.15.22:0/3898275619 learned_addr learned my addr 172.21.15.22:0/3898275619 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 --2- 172.21.15.22:0/3898275619 >> v2:172.21.15.169:3300/0 conn(0x7eff2c0cf3a0 0x7eff2c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31448700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff2c0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712328548 0 0) 0x7eff2c0ccb30 con 0x7eff2c0d1f30 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff140035d0 con 0x7eff2c0d1f30 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1895655926 0 0) 0x7eff140035d0 con 0x7eff2c0d1f30 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 msgr2=0x7eff2c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff2c0ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 >> v2:172.21.15.169:3300/0 conn(0x7eff2c0cf3a0 msgr2=0x7eff2c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 --2- 172.21.15.22:0/3898275619 >> v2:172.21.15.169:3300/0 conn(0x7eff2c0cf3a0 0x7eff2c0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0d2b60 con 0x7eff2c0d1f30 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7eff20001670 con 0x7eff2c0d1f30 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 >> v1:172.21.15.22:6789/0 conn(0x7eff2c0d1f30 legacy=0x7eff2c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14003a20 0x7eff14003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff14004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14004db0 0x7eff14005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0cfcb0 con 0x7eff14003a20 2023-06-23T18:04:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0d2880 con 0x7eff14004db0 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400ba30 con 0x7eff2c0d0730 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14003a20 0x7eff14003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff31448700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14004db0 0x7eff14005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff2bfff700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff14004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 -- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 msgr2=0x7eff14004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0d0730 0x7eff14004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 -- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14004db0 msgr2=0x7eff14005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14004db0 0x7eff14005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.578+0000 7eff30c47700 1 -- 172.21.15.22:0/3898275619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140035d0 con 0x7eff14003a20 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff30c47700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14003a20 0x7eff14003e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009b90 tx=0x7eff140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0631b5453caa8d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c005a40 con 0x7eff14003a20 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c005ba0 con 0x7eff14003a20 2023-06-23T18:04:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff2b7fe700 1 -- 172.21.15.22:0/3898275619 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00e660 con 0x7eff14003a20 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- 172.21.15.22:0/3898275619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14003a20 msgr2=0x7eff14003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 --2- 172.21.15.22:0/3898275619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14003a20 0x7eff14003e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009b90 tx=0x7eff140058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 -- 172.21.15.22:0/3898275619 reap_dead start 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- 172.21.15.22:0/3898275619 shutdown_connections 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- 172.21.15.22:0/3898275619 >> 172.21.15.22:0/3898275619 conn(0x7eff2c06a280 msgr2=0x7eff2c09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- 172.21.15.22:0/3898275619 shutdown_connections 2023-06-23T18:04:32.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- 172.21.15.22:0/3898275619 wait complete. 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 Processor -- start 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- start start 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 0x7eff2c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0c80b0 0x7eff2c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0b7280 con 0x7eff2c0c8520 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1f5cc0 con 0x7eff14003a20 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31c49700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0b6670 con 0x7eff2c0c80b0 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54516/0 (socket says 172.21.15.22:54516) 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 -- 172.21.15.22:0/3839441022 learned_addr learned my addr 172.21.15.22:0/3839441022 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff30c47700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 0x7eff2c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff2bfff700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0c80b0 0x7eff2c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 -- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0c80b0 msgr2=0x7eff2c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0c80b0 0x7eff2c0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 -- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 msgr2=0x7eff2c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 0x7eff2c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff31448700 1 -- 172.21.15.22:0/3839441022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0b6950 con 0x7eff2c0c8520 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.582+0000 7eff30c47700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 0x7eff2c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff31448700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1609c0 tx=0x7eff24011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2400e040 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2400f040 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2400fde0 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0b6c30 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c0cbc90 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c15d620 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.586+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7eff24022460 con 0x7eff2c0c8520 2023-06-23T18:04:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.590+0000 7eff297fa700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 0x7eff180687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.590+0000 7eff30c47700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 0x7eff180687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.590+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7eff240845f0 con 0x7eff2c0c8520 2023-06-23T18:04:32.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.590+0000 7eff30c47700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 0x7eff180687d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1f28f0 tx=0x7eff1c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.598+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7eff24055150 con 0x7eff2c0c8520 2023-06-23T18:04:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff2c04e4c0 con 0x7eff2c0c8520 2023-06-23T18:04:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff297fa700 1 -- 172.21.15.22:0/3839441022 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff24054ce0 con 0x7eff2c0c8520 2023-06-23T18:04:32.780 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 msgr2=0x7eff180687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 0x7eff180687d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1f28f0 tx=0x7eff1c008030 comp rx=0 tx=0).stop 2023-06-23T18:04:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 msgr2=0x7eff2c0cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1609c0 tx=0x7eff24011f80 comp rx=0 tx=0).stop 2023-06-23T18:04:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 shutdown_connections 2023-06-23T18:04:32.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff2c0c8520 0x7eff2c0cd400 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0c80b0 0x7eff2c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7eff18066320 0x7eff180687d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.766+0000 7eff31c49700 1 --2- 172.21.15.22:0/3839441022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff14003a20 0x7eff2c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.770+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 >> 172.21.15.22:0/3839441022 conn(0x7eff2c06a280 msgr2=0x7eff2c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.770+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 shutdown_connections 2023-06-23T18:04:32.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.770+0000 7eff31c49700 1 -- 172.21.15.22:0/3839441022 wait complete. 2023-06-23T18:04:32.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 Processor -- start 2023-06-23T18:04:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 -- start start 2023-06-23T18:04:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6a940715b0 0x7f6a940afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9406b5c0 con 0x7f6a940ca500 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9406c140 con 0x7f6a940b0030 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a98cc9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a940b04b0 con 0x7f6a940715b0 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6a940715b0 0x7f6a940afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6a940715b0 0x7f6a940afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54518/0 (socket says 172.21.15.22:54518) 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 -- 172.21.15.22:0/389676426 learned_addr learned my addr 172.21.15.22:0/389676426 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 -- 172.21.15.22:0/389676426 >> v1:172.21.15.169:6789/0 conn(0x7f6a940b0030 legacy=0x7f6a940c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 -- 172.21.15.22:0/389676426 >> v1:172.21.15.22:6789/0 conn(0x7f6a940ca500 legacy=0x7f6a9406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 -- 172.21.15.22:0/389676426 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9406c5d0 con 0x7f6a940715b0 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a92ffd700 1 --2- 172.21.15.22:0/389676426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940715b0 0x7f6a940afaf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e50 tx=0x7f6a8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5201596b6527364a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a927fc700 1 -- 172.21.15.22:0/389676426 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400e070 con 0x7f6a940715b0 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a927fc700 1 -- 172.21.15.22:0/389676426 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400b9c0 con 0x7f6a940715b0 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.902+0000 7f6a927fc700 1 -- 172.21.15.22:0/389676426 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400bd70 con 0x7f6a940715b0 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- 172.21.15.22:0/389676426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940715b0 msgr2=0x7f6a940afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/389676426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940715b0 0x7f6a940afaf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e50 tx=0x7f6a8400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- 172.21.15.22:0/389676426 shutdown_connections 2023-06-23T18:04:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/389676426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940715b0 0x7f6a940afaf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- 172.21.15.22:0/389676426 >> 172.21.15.22:0/389676426 conn(0x7f6a9406a5f0 msgr2=0x7f6a940caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- 172.21.15.22:0/389676426 shutdown_connections 2023-06-23T18:04:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- 172.21.15.22:0/389676426 wait complete. 2023-06-23T18:04:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 Processor -- start 2023-06-23T18:04:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.906+0000 7f6a98cc9700 1 -- start start 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a940715b0 0x7f6a941544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 0x7f6a941425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940c1a60 con 0x7f6a940b0030 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94154300 con 0x7f6a940715b0 2023-06-23T18:04:32.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941403a0 con 0x7f6a940ca500 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54522/0 (socket says 172.21.15.22:54522) 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 -- 172.21.15.22:0/1377637424 learned_addr learned my addr 172.21.15.22:0/1377637424 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a937fe700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a940715b0 0x7f6a941544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a93fff700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 0x7f6a941425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 -- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 msgr2=0x7f6a941425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 0x7f6a941425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 -- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a940715b0 msgr2=0x7f6a941544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a940715b0 0x7f6a941544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 -- 172.21.15.22:0/1377637424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84009b30 con 0x7f6a940b0030 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a93fff700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 0x7f6a941425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a92ffd700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a9406db70 tx=0x7f6a84004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84004970 con 0x7f6a940b0030 2023-06-23T18:04:32.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a840092e0 con 0x7f6a940b0030 2023-06-23T18:04:32.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a94143320 con 0x7f6a940b0030 2023-06-23T18:04:32.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8401c640 con 0x7f6a940b0030 2023-06-23T18:04:32.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.910+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a940b13b0 con 0x7f6a940b0030 2023-06-23T18:04:32.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6a8401c7a0 con 0x7f6a940b0030 2023-06-23T18:04:32.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a7bfff700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 0x7f6a800687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:32.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6a84085bb0 con 0x7f6a940b0030 2023-06-23T18:04:32.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a937fe700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 0x7f6a800687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:32.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9404e4c0 con 0x7f6a940b0030 2023-06-23T18:04:32.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.914+0000 7f6a937fe700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 0x7f6a800687d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a94155150 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:32.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:32.922+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6a84052cf0 con 0x7f6a940b0030 2023-06-23T18:04:33.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.086+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6a940b0880 con 0x7f6a940b0030 2023-06-23T18:04:33.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.086+0000 7f6a7bfff700 1 -- 172.21.15.22:0/1377637424 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6a84056310 con 0x7f6a940b0030 2023-06-23T18:04:33.100 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:33.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 msgr2=0x7f6a800687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 0x7f6a800687d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a94155150 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2023-06-23T18:04:33.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 msgr2=0x7f6a94153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a9406db70 tx=0x7f6a84004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 shutdown_connections 2023-06-23T18:04:33.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6a940ca500 0x7f6a941425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a940b0030 0x7f6a94153dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6a80066320 0x7f6a800687d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 --2- 172.21.15.22:0/1377637424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a940715b0 0x7f6a941544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 >> 172.21.15.22:0/1377637424 conn(0x7f6a9406a5f0 msgr2=0x7f6a940aada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 shutdown_connections 2023-06-23T18:04:33.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.090+0000 7f6a98cc9700 1 -- 172.21.15.22:0/1377637424 wait complete. 2023-06-23T18:04:33.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.222+0000 7f0715f44700 1 Processor -- start 2023-06-23T18:04:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.222+0000 7f0715f44700 1 -- start start 2023-06-23T18:04:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.222+0000 7f0715f44700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.222+0000 7f0715f44700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 0x7f07100d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100ccb30 con 0x7f07100d2360 2023-06-23T18:04:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07100cfcb0 con 0x7f07100d1f30 2023-06-23T18:04:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07100d2880 con 0x7f07100cf3a0 2023-06-23T18:04:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715743700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f07100d2360 0x7f071015f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:37448/0 (socket says 172.21.15.22:37448) 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:51974/0 (socket says 172.21.15.22:51974) 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715743700 1 -- 172.21.15.22:0/930823898 learned_addr learned my addr 172.21.15.22:0/930823898 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 0x7f07100d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070f7fe700 1 -- 172.21.15.22:0/930823898 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904709081 0 0) 0x7f07100ccb30 con 0x7f07100d2360 2023-06-23T18:04:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 -- 172.21.15.22:0/930823898 >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 msgr2=0x7f07100d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 --2- 172.21.15.22:0/930823898 >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 0x7f07100d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 -- 172.21.15.22:0/930823898 >> v1:172.21.15.169:6789/0 conn(0x7f07100d2360 legacy=0x7f071015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 -- 172.21.15.22:0/930823898 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100d2b60 con 0x7f07100cf3a0 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070ffff700 1 --2- 172.21.15.22:0/930823898 >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 0x7f07100d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0714f42700 1 --2- 172.21.15.22:0/930823898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009840 tx=0x7f06f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199bc9719f1dece0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070f7fe700 1 -- 172.21.15.22:0/930823898 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8005e50 con 0x7f07100cf3a0 2023-06-23T18:04:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070f7fe700 1 -- 172.21.15.22:0/930823898 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f8006030 con 0x7f07100cf3a0 2023-06-23T18:04:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f070f7fe700 1 -- 172.21.15.22:0/930823898 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80063c0 con 0x7f07100cf3a0 2023-06-23T18:04:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- 172.21.15.22:0/930823898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 msgr2=0x7f07100cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 --2- 172.21.15.22:0/930823898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009840 tx=0x7f06f8009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- 172.21.15.22:0/930823898 shutdown_connections 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 --2- 172.21.15.22:0/930823898 >> v2:172.21.15.169:3300/0 conn(0x7f07100d1f30 0x7f07100d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 --2- 172.21.15.22:0/930823898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100cf3a0 0x7f07100cf770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- 172.21.15.22:0/930823898 >> 172.21.15.22:0/930823898 conn(0x7f071006a280 msgr2=0x7f071009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- 172.21.15.22:0/930823898 shutdown_connections 2023-06-23T18:04:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.226+0000 7f0715f44700 1 -- 172.21.15.22:0/930823898 wait complete. 2023-06-23T18:04:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 Processor -- start 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 -- start start 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07100cf3a0 0x7f071015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100d2360 0x7f07100d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071015dce0 con 0x7f07100d1f30 2023-06-23T18:04:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100b7ef0 con 0x7f07100cf3a0 2023-06-23T18:04:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715f44700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071015d0d0 con 0x7f07100d2360 2023-06-23T18:04:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54528/0 (socket says 172.21.15.22:54528) 2023-06-23T18:04:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 -- 172.21.15.22:0/93347252 learned_addr learned my addr 172.21.15.22:0/93347252 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0714f42700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07100cf3a0 0x7f071015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f0715743700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100d2360 0x7f07100d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 -- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100d2360 msgr2=0x7f07100d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100d2360 0x7f07100d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 -- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07100cf3a0 msgr2=0x7f071015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07100cf3a0 0x7f071015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.230+0000 7f070ffff700 1 -- 172.21.15.22:0/93347252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000094b0 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070ffff700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f070000ebe0 tx=0x7f070000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000c850 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f8009450 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000c9b0 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000cd40 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071015d690 con 0x7f07100d1f30 2023-06-23T18:04:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071015de50 con 0x7f07100d1f30 2023-06-23T18:04:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0700013470 con 0x7f07100d1f30 2023-06-23T18:04:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.234+0000 7f070d7fa700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 0x7f06f00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.238+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0700084120 con 0x7f07100d1f30 2023-06-23T18:04:33.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.238+0000 7f0714f42700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 0x7f06f00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.238+0000 7f0714f42700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 0x7f06f00687a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009b90 tx=0x7f06f8008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.242+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f07000511f0 con 0x7f07100d1f30 2023-06-23T18:04:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.402+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f071004e4c0 con 0x7f07100d1f30 2023-06-23T18:04:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.402+0000 7f070d7fa700 1 -- 172.21.15.22:0/93347252 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f070000cea0 con 0x7f07100d1f30 2023-06-23T18:04:33.417 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 msgr2=0x7f06f00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 0x7f06f00687a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009b90 tx=0x7f06f8008070 comp rx=0 tx=0).stop 2023-06-23T18:04:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 msgr2=0x7f07100b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f070000ebe0 tx=0x7f070000eef0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 shutdown_connections 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07100d2360 0x7f07100d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07100d1f30 0x7f07100b79b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f06f00662f0 0x7f06f00687a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 --2- 172.21.15.22:0/93347252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07100cf3a0 0x7f071015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 >> 172.21.15.22:0/93347252 conn(0x7f071006a280 msgr2=0x7f071015f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 shutdown_connections 2023-06-23T18:04:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.406+0000 7f0715f44700 1 -- 172.21.15.22:0/93347252 wait complete. 2023-06-23T18:04:33.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:33.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.530+0000 7f2c664ea700 1 Processor -- start 2023-06-23T18:04:33.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.530+0000 7f2c664ea700 1 -- start start 2023-06-23T18:04:33.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2c6006f660 0x7f2c600c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 0x7f2c600c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6006c6e0 con 0x7f2c6015bf10 2023-06-23T18:04:33.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6006cf70 con 0x7f2c6006f660 2023-06-23T18:04:33.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600cddf0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c654e8700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2c6015bf10 0x7f2c6006bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58376/0 (socket says 172.21.15.22:58376) 2023-06-23T18:04:33.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c654e8700 1 -- 172.21.15.22:0/3943521292 learned_addr learned my addr 172.21.15.22:0/3943521292 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:33.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3943521292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 0x7f2c600c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c64ce7700 1 --2- 172.21.15.22:0/3943521292 >> v2:172.21.15.169:3300/0 conn(0x7f2c6006f660 0x7f2c600c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c57fff700 1 -- 172.21.15.22:0/3943521292 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719613924 0 0) 0x7f2c6006c6e0 con 0x7f2c6015bf10 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c57fff700 1 -- 172.21.15.22:0/3943521292 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c4c0035d0 con 0x7f2c6015bf10 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3943521292 >> v2:172.21.15.169:3300/0 conn(0x7f2c6006f660 msgr2=0x7f2c600c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3943521292 >> v2:172.21.15.169:3300/0 conn(0x7f2c6006f660 0x7f2c600c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3943521292 >> v1:172.21.15.22:6789/0 conn(0x7f2c6015bf10 legacy=0x7f2c6006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3943521292 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6006c9b0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3943521292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 0x7f2c600c8f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d90 tx=0x7f2c5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29259931a2434a50 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c57fff700 1 -- 172.21.15.22:0/3943521292 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800e070 con 0x7f2c6015c2e0 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c57fff700 1 -- 172.21.15.22:0/3943521292 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800bb20 con 0x7f2c6015c2e0 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c57fff700 1 -- 172.21.15.22:0/3943521292 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58012430 con 0x7f2c6015c2e0 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- 172.21.15.22:0/3943521292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 msgr2=0x7f2c600c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3943521292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 0x7f2c600c8f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d90 tx=0x7f2c5800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- 172.21.15.22:0/3943521292 shutdown_connections 2023-06-23T18:04:33.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3943521292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015c2e0 0x7f2c600c8f30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3943521292 >> v2:172.21.15.169:3300/0 conn(0x7f2c6006f660 0x7f2c600c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- 172.21.15.22:0/3943521292 >> 172.21.15.22:0/3943521292 conn(0x7f2c6006a5f0 msgr2=0x7f2c600caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- 172.21.15.22:0/3943521292 shutdown_connections 2023-06-23T18:04:33.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.534+0000 7f2c664ea700 1 -- 172.21.15.22:0/3943521292 wait complete. 2023-06-23T18:04:33.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 Processor -- start 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- start start 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6006f660 0x7f2c600b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015bf10 0x7f2c600c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6009b880 con 0x7f2c6015c2e0 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60099b00 con 0x7f2c6006f660 2023-06-23T18:04:33.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60099de0 con 0x7f2c6015bf10 2023-06-23T18:04:33.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54534/0 (socket says 172.21.15.22:54534) 2023-06-23T18:04:33.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3338196093 learned_addr learned my addr 172.21.15.22:0/3338196093 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:33.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c64ce7700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015bf10 0x7f2c600c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c654e8700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6006f660 0x7f2c600b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015bf10 msgr2=0x7f2c600c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015bf10 0x7f2c600c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6006f660 msgr2=0x7f2c600b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6006f660 0x7f2c600b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 -- 172.21.15.22:0/3338196093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58009a70 con 0x7f2c6015c2e0 2023-06-23T18:04:33.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c65ce9700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c58014040 tx=0x7f2c58004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58012430 con 0x7f2c6015c2e0 2023-06-23T18:04:33.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58020070 con 0x7f2c6015c2e0 2023-06-23T18:04:33.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6009a0c0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58003e10 con 0x7f2c6015c2e0 2023-06-23T18:04:33.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.538+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6009a680 con 0x7f2c6015c2e0 2023-06-23T18:04:33.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6004e4c0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2c58004040 con 0x7f2c6015c2e0 2023-06-23T18:04:33.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c55ffb700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 0x7f2c440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c58022070 con 0x7f2c6015c2e0 2023-06-23T18:04:33.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c654e8700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 0x7f2c440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.542+0000 7f2c654e8700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 0x7f2c440687a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c50007fd0 tx=0x7f2c50007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.550+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2c58052d70 con 0x7f2c6015c2e0 2023-06-23T18:04:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.714+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c600b3ff0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.714+0000 7f2c55ffb700 1 -- 172.21.15.22:0/3338196093 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c5802c0a0 con 0x7f2c6015c2e0 2023-06-23T18:04:33.730 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 msgr2=0x7f2c440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 0x7f2c440687a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c50007fd0 tx=0x7f2c50007de0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 msgr2=0x7f2c6009aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c58014040 tx=0x7f2c58004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 shutdown_connections 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c6015c2e0 0x7f2c6009aba0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6015bf10 0x7f2c600c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2c440662f0 0x7f2c440687a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 --2- 172.21.15.22:0/3338196093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6006f660 0x7f2c600b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 >> 172.21.15.22:0/3338196093 conn(0x7f2c6006a5f0 msgr2=0x7f2c600ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 shutdown_connections 2023-06-23T18:04:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.718+0000 7f2c664ea700 1 -- 172.21.15.22:0/3338196093 wait complete. 2023-06-23T18:04:33.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:33.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 Processor -- start 2023-06-23T18:04:33.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 -- start start 2023-06-23T18:04:33.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d900d1f30 0x7f7d900d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900ccb30 con 0x7f7d900cf3a0 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900d2880 con 0x7f7d900d2360 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.854+0000 7f7d8ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d900d2b60 con 0x7f7d900d1f30 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d900d1f30 0x7f7d900d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d900d1f30 0x7f7d900d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54536/0 (socket says 172.21.15.22:54536) 2023-06-23T18:04:33.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/2663824682 learned_addr learned my addr 172.21.15.22:0/2663824682 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:33.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/2663824682 >> v1:172.21.15.169:6789/0 conn(0x7f7d900d2360 legacy=0x7f7d900ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/2663824682 >> v1:172.21.15.22:6789/0 conn(0x7f7d900cf3a0 legacy=0x7f7d900cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8dffb700 1 -- 172.21.15.22:0/2663824682 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107003146 0 0) 0x7f7d900ccb30 con 0x7f7d900cf3a0 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/2663824682 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900d2e40 con 0x7f7d900d1f30 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8e7fc700 1 --2- 172.21.15.22:0/2663824682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900d0020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009910 tx=0x7f7d78009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92986dd46045ded7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8dffb700 1 -- 172.21.15.22:0/2663824682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ba60 con 0x7f7d900d1f30 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8dffb700 1 -- 172.21.15.22:0/2663824682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800bbc0 con 0x7f7d900d1f30 2023-06-23T18:04:33.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8dffb700 1 -- 172.21.15.22:0/2663824682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78009e60 con 0x7f7d900d1f30 2023-06-23T18:04:33.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 -- 172.21.15.22:0/2663824682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 msgr2=0x7f7d900d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/2663824682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900d0020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009910 tx=0x7f7d78009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:33.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 -- 172.21.15.22:0/2663824682 shutdown_connections 2023-06-23T18:04:33.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/2663824682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900d0020 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 -- 172.21.15.22:0/2663824682 >> 172.21.15.22:0/2663824682 conn(0x7f7d9006a280 msgr2=0x7f7d9009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:33.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 -- 172.21.15.22:0/2663824682 shutdown_connections 2023-06-23T18:04:33.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.858+0000 7f7d8ffff700 1 -- 172.21.15.22:0/2663824682 wait complete. 2023-06-23T18:04:33.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 Processor -- start 2023-06-23T18:04:33.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- start start 2023-06-23T18:04:33.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cf3a0 0x7f7d9009f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 0x7f7d900b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9009feb0 con 0x7f7d900d1f30 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900b55d0 con 0x7f7d900d2360 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900b58b0 con 0x7f7d900cf3a0 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54548/0 (socket says 172.21.15.22:54548) 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/3345003653 learned_addr learned my addr 172.21.15.22:0/3345003653 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8effd700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cf3a0 0x7f7d9009f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8f7fe700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 0x7f7d900b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cf3a0 msgr2=0x7f7d9009f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cf3a0 0x7f7d9009f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 msgr2=0x7f7d900b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 0x7f7d900b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 -- 172.21.15.22:0/3345003653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009570 con 0x7f7d900d1f30 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8f7fe700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 0x7f7d900b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8e7fc700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000c00 tx=0x7f7d78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78011ab0 con 0x7f7d900d1f30 2023-06-23T18:04:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7801f070 con 0x7f7d900d1f30 2023-06-23T18:04:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d900b5b90 con 0x7f7d900d1f30 2023-06-23T18:04:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78011e30 con 0x7f7d900d1f30 2023-06-23T18:04:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.862+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d900b6150 con 0x7f7d900d1f30 2023-06-23T18:04:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.866+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9004e4c0 con 0x7f7d900d1f30 2023-06-23T18:04:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.866+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7d78026480 con 0x7f7d900d1f30 2023-06-23T18:04:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.866+0000 7f7d877fe700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 0x7f7d700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.866+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7d78021070 con 0x7f7d900d1f30 2023-06-23T18:04:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.866+0000 7f7d8effd700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 0x7f7d700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:33.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.870+0000 7f7d8effd700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 0x7f7d700687a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d80007aa0 tx=0x7f7d80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:33.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:33.878+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7d78052cf0 con 0x7f7d900d1f30 2023-06-23T18:04:34.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d900b18a0 con 0x7f7d900d1f30 2023-06-23T18:04:34.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d877fe700 1 -- 172.21.15.22:0/3345003653 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d780150a0 con 0x7f7d900d1f30 2023-06-23T18:04:34.051 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:34.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 msgr2=0x7f7d700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 0x7f7d700687a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d80007aa0 tx=0x7f7d80009040 comp rx=0 tx=0).stop 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 msgr2=0x7f7d900b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000c00 tx=0x7f7d78004770 comp rx=0 tx=0).stop 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 shutdown_connections 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d700662f0 0x7f7d700687a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d900d2360 0x7f7d900b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d900d1f30 0x7f7d900b2580 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 --2- 172.21.15.22:0/3345003653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cf3a0 0x7f7d9009f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 >> 172.21.15.22:0/3345003653 conn(0x7f7d9006a280 msgr2=0x7f7d900b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 shutdown_connections 2023-06-23T18:04:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.038+0000 7f7d8ffff700 1 -- 172.21.15.22:0/3345003653 wait complete. 2023-06-23T18:04:34.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 Processor -- start 2023-06-23T18:04:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- start start 2023-06-23T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2300c0fe0 0x7fe2300c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2300c18f0 con 0x7fe230160420 2023-06-23T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe230163ec0 con 0x7fe2300cc380 2023-06-23T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2301641a0 con 0x7fe2300c0fe0 2023-06-23T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2300c0fe0 0x7fe2300c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe2277fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe230160420 0x7fe2300cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58398/0 (socket says 172.21.15.22:58398) 2023-06-23T18:04:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe2277fe700 1 -- 172.21.15.22:0/2221445983 learned_addr learned my addr 172.21.15.22:0/2221445983 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22ffff700 1 -- 172.21.15.22:0/2221445983 >> v1:172.21.15.169:6789/0 conn(0x7fe2300cc380 legacy=0x7fe2300c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22ffff700 1 -- 172.21.15.22:0/2221445983 >> v1:172.21.15.22:6789/0 conn(0x7fe230160420 legacy=0x7fe2300cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22ffff700 1 -- 172.21.15.22:0/2221445983 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230164480 con 0x7fe2300c0fe0 2023-06-23T18:04:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22ffff700 1 --2- 172.21.15.22:0/2221445983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2300c0fe0 0x7fe2300c13b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe220009850 tx=0x7fe220009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df2b7943c1c6ba server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22f7fe700 1 -- 172.21.15.22:0/2221445983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220011830 con 0x7fe2300c0fe0 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22f7fe700 1 -- 172.21.15.22:0/2221445983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe220011990 con 0x7fe2300c0fe0 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe22f7fe700 1 -- 172.21.15.22:0/2221445983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220011d20 con 0x7fe2300c0fe0 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- 172.21.15.22:0/2221445983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2300c0fe0 msgr2=0x7fe2300c13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 --2- 172.21.15.22:0/2221445983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2300c0fe0 0x7fe2300c13b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe220009850 tx=0x7fe220009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- 172.21.15.22:0/2221445983 shutdown_connections 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 --2- 172.21.15.22:0/2221445983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2300c0fe0 0x7fe2300c13b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- 172.21.15.22:0/2221445983 >> 172.21.15.22:0/2221445983 conn(0x7fe23006a5f0 msgr2=0x7fe2300caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- 172.21.15.22:0/2221445983 shutdown_connections 2023-06-23T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.170+0000 7fe235369700 1 -- 172.21.15.22:0/2221445983 wait complete. 2023-06-23T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 Processor -- start 2023-06-23T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 -- start start 2023-06-23T18:04:34.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2300c0fe0 0x7fe23015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 0x7fe2300abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23015a6e0 con 0x7fe230160420 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300997b0 con 0x7fe2300cc380 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe235369700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe230099a90 con 0x7fe2300c0fe0 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54570/0 (socket says 172.21.15.22:54570) 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 -- 172.21.15.22:0/2555388237 learned_addr learned my addr 172.21.15.22:0/2555388237 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:34.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe2277fe700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 0x7fe2300abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe22ffff700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2300c0fe0 0x7fe23015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 -- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2300c0fe0 msgr2=0x7fe23015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2300c0fe0 0x7fe23015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 -- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 msgr2=0x7fe2300abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 0x7fe2300abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe234b68700 1 -- 172.21.15.22:0/2555388237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200094b0 con 0x7fe230160420 2023-06-23T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.174+0000 7fe2277fe700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 0x7fe2300abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe234b68700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe22800cc50 tx=0x7fe22800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800ad80 con 0x7fe230160420 2023-06-23T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800ce50 con 0x7fe230160420 2023-06-23T18:04:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe230099d70 con 0x7fe230160420 2023-06-23T18:04:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228003c60 con 0x7fe230160420 2023-06-23T18:04:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23015afa0 con 0x7fe230160420 2023-06-23T18:04:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23015b220 con 0x7fe230160420 2023-06-23T18:04:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.178+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe2280050d0 con 0x7fe230160420 2023-06-23T18:04:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.182+0000 7fe22d7fa700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 0x7fe2100687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.182+0000 7fe22ffff700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 0x7fe2100687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.182+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe228084180 con 0x7fe230160420 2023-06-23T18:04:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.182+0000 7fe22ffff700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 0x7fe2100687a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe220009fd0 tx=0x7fe220008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.186+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe2280512d0 con 0x7fe230160420 2023-06-23T18:04:34.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.358+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe23004e4c0 con 0x7fe230160420 2023-06-23T18:04:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.358+0000 7fe22d7fa700 1 -- 172.21.15.22:0/2555388237 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe2280548f0 con 0x7fe230160420 2023-06-23T18:04:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 msgr2=0x7fe2100687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 0x7fe2100687a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe220009fd0 tx=0x7fe220008030 comp rx=0 tx=0).stop 2023-06-23T18:04:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 msgr2=0x7fe2300ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe22800cc50 tx=0x7fe22800a340 comp rx=0 tx=0).stop 2023-06-23T18:04:34.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 shutdown_connections 2023-06-23T18:04:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230160420 0x7fe2300ac4e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe2100662f0 0x7fe2100687a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2300cc380 0x7fe2300abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 --2- 172.21.15.22:0/2555388237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2300c0fe0 0x7fe23015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 >> 172.21.15.22:0/2555388237 conn(0x7fe23006a5f0 msgr2=0x7fe23006c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 shutdown_connections 2023-06-23T18:04:34.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.362+0000 7fe235369700 1 -- 172.21.15.22:0/2555388237 wait complete. 2023-06-23T18:04:34.377 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:34.388 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2023-06-23T18:04:34.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:34.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-06-23T18:04:34.404 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:34.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:04:34.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:34.406 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-23T18:04:34.413 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:34.413 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:34.421 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:34.421 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:34.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 Processor -- start 2023-06-23T18:04:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 -- start start 2023-06-23T18:04:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34900cc9e0 0x7f34900d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34900b7ec0 con 0x7f34900b7af0 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34900b1ad0 con 0x7f34900cc9e0 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.558+0000 7f348f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34900d0a10 con 0x7f34900cf480 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34900cc9e0 0x7f34900d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34900cc9e0 0x7f34900d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54578/0 (socket says 172.21.15.22:54578) 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 -- 172.21.15.22:0/2949765003 learned_addr learned my addr 172.21.15.22:0/2949765003 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:34.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348dd9b700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 -- 172.21.15.22:0/2949765003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 msgr2=0x7f34900ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 -- 172.21.15.22:0/2949765003 >> v1:172.21.15.22:6789/0 conn(0x7f34900b7af0 legacy=0x7f34900ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348d59a700 1 -- 172.21.15.22:0/2949765003 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715182289 0 0) 0x7f34900b7ec0 con 0x7f34900b7af0 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 -- 172.21.15.22:0/2949765003 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900d0cb0 con 0x7f34900cc9e0 2023-06-23T18:04:34.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348ed9d700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900d18e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3488009910 tx=0x7f3488009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8794a6be424dc389 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348d59a700 1 -- 172.21.15.22:0/2949765003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488011830 con 0x7f34900cc9e0 2023-06-23T18:04:34.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348d59a700 1 -- 172.21.15.22:0/2949765003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488011990 con 0x7f34900cc9e0 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348d59a700 1 -- 172.21.15.22:0/2949765003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488011d40 con 0x7f34900cc9e0 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 -- 172.21.15.22:0/2949765003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 msgr2=0x7f34900d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900d18e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3488009910 tx=0x7f3488009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 -- 172.21.15.22:0/2949765003 shutdown_connections 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900d18e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 --2- 172.21.15.22:0/2949765003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 -- 172.21.15.22:0/2949765003 >> 172.21.15.22:0/2949765003 conn(0x7f3490069960 msgr2=0x7f3490069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 -- 172.21.15.22:0/2949765003 shutdown_connections 2023-06-23T18:04:34.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.562+0000 7f348f59e700 1 -- 172.21.15.22:0/2949765003 wait complete. 2023-06-23T18:04:34.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 Processor -- start 2023-06-23T18:04:34.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- start start 2023-06-23T18:04:34.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 0x7f349015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349015ead0 con 0x7f34900cc9e0 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349015a880 con 0x7f34900b7af0 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349015de50 con 0x7f34900cf480 2023-06-23T18:04:34.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54580/0 (socket says 172.21.15.22:54580) 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 -- 172.21.15.22:0/3058412474 learned_addr learned my addr 172.21.15.22:0/3058412474 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348e59c700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 0x7f349015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348ed9d700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 -- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 msgr2=0x7f34900cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 -- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 msgr2=0x7f349015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 0x7f349015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 -- 172.21.15.22:0/3058412474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009570 con 0x7f34900cc9e0 2023-06-23T18:04:34.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348ed9d700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:34.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348e59c700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 0x7f349015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:34.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348dd9b700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f347800efc0 tx=0x7f347800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478009b80 con 0x7f34900cc9e0 2023-06-23T18:04:34.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800f040 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349015e130 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478014480 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.566+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349015e6f0 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.570+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349004e4c0 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.570+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f34780145e0 con 0x7f34900cc9e0 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.570+0000 7f3486ffd700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 0x7f34700687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.570+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3478084c30 con 0x7f34900cc9e0 2023-06-23T18:04:34.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.570+0000 7f348e59c700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 0x7f34700687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.574+0000 7f348e59c700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 0x7f34700687a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3480000f60 tx=0x7f3480009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.578+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3478055820 con 0x7f34900cc9e0 2023-06-23T18:04:34.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.742+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34900b4740 con 0x7f34900cc9e0 2023-06-23T18:04:34.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.746+0000 7f3486ffd700 1 -- 172.21.15.22:0/3058412474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f34780553b0 con 0x7f34900cc9e0 2023-06-23T18:04:34.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:34.761 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:34.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 msgr2=0x7f34700687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 0x7f34700687a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3480000f60 tx=0x7f3480009040 comp rx=0 tx=0).stop 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 msgr2=0x7f34900ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f347800efc0 tx=0x7f347800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 shutdown_connections 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34900cf480 0x7f34900cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34900cc9e0 0x7f34900ca720 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f34700662f0 0x7f34700687a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 --2- 172.21.15.22:0/3058412474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34900b7af0 0x7f349015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 >> 172.21.15.22:0/3058412474 conn(0x7f3490069960 msgr2=0x7f349015b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 shutdown_connections 2023-06-23T18:04:34.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.750+0000 7f348f59e700 1 -- 172.21.15.22:0/3058412474 wait complete. 2023-06-23T18:04:34.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:34.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:34.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 Processor -- start 2023-06-23T18:04:34.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 -- start start 2023-06-23T18:04:34.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15e415bdd0 0x7f15e4160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15e406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e406b160 con 0x7f15e406c150 2023-06-23T18:04:34.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e406bb30 con 0x7f15e415bdd0 2023-06-23T18:04:34.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15ea17d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40b2c80 con 0x7f15e40cfa50 2023-06-23T18:04:34.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f15e406c150 0x7f15e40cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58422/0 (socket says 172.21.15.22:58422) 2023-06-23T18:04:34.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 -- 172.21.15.22:0/4067802083 learned_addr learned my addr 172.21.15.22:0/4067802083 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:34.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e997c700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15e406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e917b700 1 --2- 172.21.15.22:0/4067802083 >> v2:172.21.15.169:3300/0 conn(0x7f15e415bdd0 0x7f15e4160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232585362 0 0) 0x7f15e406b160 con 0x7f15e406c150 2023-06-23T18:04:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d00035d0 con 0x7f15e406c150 2023-06-23T18:04:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4236994785 0 0) 0x7f15d00035d0 con 0x7f15e406c150 2023-06-23T18:04:34.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 msgr2=0x7f15e406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:34.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15e406b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 >> v2:172.21.15.169:3300/0 conn(0x7f15e415bdd0 msgr2=0x7f15e4160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:34.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 --2- 172.21.15.22:0/4067802083 >> v2:172.21.15.169:3300/0 conn(0x7f15e415bdd0 0x7f15e4160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e406c9d0 con 0x7f15e406c150 2023-06-23T18:04:34.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f15e0001670 con 0x7f15e406c150 2023-06-23T18:04:34.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 >> v1:172.21.15.22:6789/0 conn(0x7f15e406c150 legacy=0x7f15e40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15d0003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d0003fa0 0x7f15d0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d0004db0 0x7f15d0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e406bb30 con 0x7f15d0003fa0 2023-06-23T18:04:34.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40b2c80 con 0x7f15d0004db0 2023-06-23T18:04:34.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d000ba60 con 0x7f15e40cfa50 2023-06-23T18:04:34.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d0003fa0 0x7f15d0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e997c700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d0004db0 0x7f15d0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e917b700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15d0003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 -- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 msgr2=0x7f15d0003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:34.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15d0003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 -- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d0004db0 msgr2=0x7f15d0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:34.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d0004db0 0x7f15d0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e897a700 1 -- 172.21.15.22:0/4067802083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00035d0 con 0x7f15d0003fa0 2023-06-23T18:04:34.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.894+0000 7f15e917b700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e40cfa50 0x7f15d0003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:34.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e897a700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d0003fa0 0x7f15d0004870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f15e406a920 tx=0x7f15d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49749f5d69443b79 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0007c80 con 0x7f15d0003fa0 2023-06-23T18:04:34.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e000ce00 con 0x7f15d0003fa0 2023-06-23T18:04:34.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15dbfff700 1 -- 172.21.15.22:0/4067802083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005bc0 con 0x7f15d0003fa0 2023-06-23T18:04:34.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- 172.21.15.22:0/4067802083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d0003fa0 msgr2=0x7f15d0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 --2- 172.21.15.22:0/4067802083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d0003fa0 0x7f15d0004870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f15e406a920 tx=0x7f15d00058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e997c700 1 -- 172.21.15.22:0/4067802083 reap_dead start 2023-06-23T18:04:34.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- 172.21.15.22:0/4067802083 shutdown_connections 2023-06-23T18:04:34.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- 172.21.15.22:0/4067802083 >> 172.21.15.22:0/4067802083 conn(0x7f15e4069960 msgr2=0x7f15e4069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:34.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- 172.21.15.22:0/4067802083 shutdown_connections 2023-06-23T18:04:34.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- 172.21.15.22:0/4067802083 wait complete. 2023-06-23T18:04:34.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 Processor -- start 2023-06-23T18:04:34.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- start start 2023-06-23T18:04:34.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d0003fa0 0x7f15e40c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e414dbd0 0x7f15e40c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e414b350 con 0x7f15e40c33e0 2023-06-23T18:04:34.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c5d70 con 0x7f15e414dbd0 2023-06-23T18:04:34.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15ea17d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c6010 con 0x7f15d0003fa0 2023-06-23T18:04:34.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e997c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e997c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54604/0 (socket says 172.21.15.22:54604) 2023-06-23T18:04:34.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e997c700 1 -- 172.21.15.22:0/609941902 learned_addr learned my addr 172.21.15.22:0/609941902 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:34.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e897a700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e414dbd0 0x7f15e40c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.898+0000 7f15e917b700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d0003fa0 0x7f15e40c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 -- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d0003fa0 msgr2=0x7f15e40c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d0003fa0 0x7f15e40c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 -- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e414dbd0 msgr2=0x7f15e40c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:34.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e414dbd0 0x7f15e40c2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:34.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 -- 172.21.15.22:0/609941902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40c62f0 con 0x7f15e40c33e0 2023-06-23T18:04:34.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15e997c700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e415f6a0 tx=0x7f15dc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00e040 con 0x7f15e40c33e0 2023-06-23T18:04:34.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e40c65d0 con 0x7f15e40c33e0 2023-06-23T18:04:34.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc00f040 con 0x7f15e40c33e0 2023-06-23T18:04:34.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00fc40 con 0x7f15e40c33e0 2023-06-23T18:04:34.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e415c780 con 0x7f15e40c33e0 2023-06-23T18:04:34.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e40c67c0 con 0x7f15e40c33e0 2023-06-23T18:04:34.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.902+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f15dc022460 con 0x7f15e40c33e0 2023-06-23T18:04:34.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.906+0000 7f15d9ffb700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 0x7f15c80687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:34.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.906+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f15dc084490 con 0x7f15e40c33e0 2023-06-23T18:04:34.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.906+0000 7f15e917b700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 0x7f15c80687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:34.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.906+0000 7f15e917b700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 0x7f15c80687d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bd0 tx=0x7f15d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:34.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:34.914+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f15dc054ff0 con 0x7f15e40c33e0 2023-06-23T18:04:35.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.074+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15e404e4c0 con 0x7f15e40c33e0 2023-06-23T18:04:35.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.074+0000 7f15d9ffb700 1 -- 172.21.15.22:0/609941902 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15dc054b80 con 0x7f15e40c33e0 2023-06-23T18:04:35.089 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:35.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 msgr2=0x7f15c80687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 0x7f15c80687d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bd0 tx=0x7f15d4008040 comp rx=0 tx=0).stop 2023-06-23T18:04:35.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 msgr2=0x7f15e415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e415f6a0 tx=0x7f15dc011ed0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 shutdown_connections 2023-06-23T18:04:35.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15e40c33e0 0x7f15e415c240 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f15c8066320 0x7f15c80687d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e414dbd0 0x7f15e40c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 --2- 172.21.15.22:0/609941902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d0003fa0 0x7f15e40c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 >> 172.21.15.22:0/609941902 conn(0x7f15e4069960 msgr2=0x7f15e40b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 shutdown_connections 2023-06-23T18:04:35.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.078+0000 7f15ea17d700 1 -- 172.21.15.22:0/609941902 wait complete. 2023-06-23T18:04:35.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.210+0000 7f848d57c700 1 Processor -- start 2023-06-23T18:04:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 -- start start 2023-06-23T18:04:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84880cf3a0 0x7f84880cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84880ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84880ccb30 con 0x7f84880d1f30 2023-06-23T18:04:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880cfcb0 con 0x7f84880cf3a0 2023-06-23T18:04:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848d57c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880d2880 con 0x7f84880d0730 2023-06-23T18:04:35.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f847f7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f84880d1f30 0x7f84880d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58438/0 (socket says 172.21.15.22:58438) 2023-06-23T18:04:35.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f847f7fe700 1 -- 172.21.15.22:0/1018418570 learned_addr learned my addr 172.21.15.22:0/1018418570 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 --2- 172.21.15.22:0/1018418570 >> v2:172.21.15.169:3300/0 conn(0x7f84880cf3a0 0x7f84880cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f848cd7b700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84880ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f84877fe700 1 -- 172.21.15.22:0/1018418570 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136840216 0 0) 0x7f84880ccb30 con 0x7f84880d1f30 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f84877fe700 1 -- 172.21.15.22:0/1018418570 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84740035d0 con 0x7f84880d1f30 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 -- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 msgr2=0x7f84880ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84880ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 -- 172.21.15.22:0/1018418570 >> v1:172.21.15.22:6789/0 conn(0x7f84880d1f30 legacy=0x7f84880d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 -- 172.21.15.22:0/1018418570 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880d2b60 con 0x7f84880cf3a0 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f8487fff700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880cf770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8478009840 tx=0x7f8478009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab4ad5eecf38092 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f84877fe700 1 -- 172.21.15.22:0/1018418570 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478005e50 con 0x7f84880cf3a0 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f84877fe700 1 -- 172.21.15.22:0/1018418570 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478006030 con 0x7f84880cf3a0 2023-06-23T18:04:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.214+0000 7f84877fe700 1 -- 172.21.15.22:0/1018418570 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780063a0 con 0x7f84880cf3a0 2023-06-23T18:04:35.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 msgr2=0x7f84880cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880cf770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8478009840 tx=0x7f8478009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:35.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- 172.21.15.22:0/1018418570 shutdown_connections 2023-06-23T18:04:35.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84880ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- 172.21.15.22:0/1018418570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880cf770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- 172.21.15.22:0/1018418570 >> 172.21.15.22:0/1018418570 conn(0x7f848806a280 msgr2=0x7f848809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- 172.21.15.22:0/1018418570 shutdown_connections 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- 172.21.15.22:0/1018418570 wait complete. 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 Processor -- start 2023-06-23T18:04:35.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- start start 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84881f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881f28d0 con 0x7f84880d1f30 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881f2310 con 0x7f84880cf3a0 2023-06-23T18:04:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848d57c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84881f25f0 con 0x7f84880d0730 2023-06-23T18:04:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54608/0 (socket says 172.21.15.22:54608) 2023-06-23T18:04:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 -- 172.21.15.22:0/3108351669 learned_addr learned my addr 172.21.15.22:0/3108351669 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f847f7fe700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84881f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f8487fff700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 -- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 msgr2=0x7f84881f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84881f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 -- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 msgr2=0x7f84880c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.218+0000 7f848cd7b700 1 -- 172.21.15.22:0/3108351669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478009450 con 0x7f84880d1f30 2023-06-23T18:04:35.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f848cd7b700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f848000efc0 tx=0x7f848000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000cc90 con 0x7f84880d1f30 2023-06-23T18:04:35.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000f040 con 0x7f84880d1f30 2023-06-23T18:04:35.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800146c0 con 0x7f84880d1f30 2023-06-23T18:04:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84881f2c20 con 0x7f84880d1f30 2023-06-23T18:04:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880c85d0 con 0x7f84880d1f30 2023-06-23T18:04:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848804e4c0 con 0x7f84880d1f30 2023-06-23T18:04:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.222+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8480014820 con 0x7f84880d1f30 2023-06-23T18:04:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.226+0000 7f84857fa700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 0x7f84680687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.226+0000 7f8487fff700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 0x7f84680687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.226+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8480084ac0 con 0x7f84880d1f30 2023-06-23T18:04:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.226+0000 7f8487fff700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 0x7f84680687d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f8478008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.234+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8480051ca0 con 0x7f84880d1f30 2023-06-23T18:04:35.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.398+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f84880d2e10 con 0x7f84880d1f30 2023-06-23T18:04:35.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f84857fa700 1 -- 172.21.15.22:0/3108351669 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8480009ca0 con 0x7f84880d1f30 2023-06-23T18:04:35.414 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 msgr2=0x7f84680687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 0x7f84680687d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f8478008070 comp rx=0 tx=0).stop 2023-06-23T18:04:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 msgr2=0x7f848815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f848000efc0 tx=0x7f848000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 shutdown_connections 2023-06-23T18:04:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84880d1f30 0x7f848815c7d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d0730 0x7f84881f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8468066320 0x7f84680687d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 --2- 172.21.15.22:0/3108351669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84880cf3a0 0x7f84880c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 >> 172.21.15.22:0/3108351669 conn(0x7f848806a280 msgr2=0x7f848806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 shutdown_connections 2023-06-23T18:04:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.402+0000 7f848d57c700 1 -- 172.21.15.22:0/3108351669 wait complete. 2023-06-23T18:04:35.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.534+0000 7f493359e700 1 Processor -- start 2023-06-23T18:04:35.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.534+0000 7f493359e700 1 -- start start 2023-06-23T18:04:35.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.534+0000 7f493359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49340cca20 0x7f49340d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340d1d40 con 0x7f49340d21d0 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340cf130 con 0x7f49340b1920 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49340b1cf0 con 0x7f49340cca20 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f49340d21d0 0x7f49340d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58454/0 (socket says 172.21.15.22:58454) 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 -- 172.21.15.22:0/3506897992 learned_addr learned my addr 172.21.15.22:0/3506897992 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4931d9b700 1 --2- 172.21.15.22:0/3506897992 >> v2:172.21.15.169:3300/0 conn(0x7f49340cca20 0x7f49340d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037861948 0 0) 0x7f49340d1d40 con 0x7f49340d21d0 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49200035d0 con 0x7f49340d21d0 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186862809 0 0) 0x7f49340cf130 con 0x7f49340b1920 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49340d1d40 con 0x7f49340b1920 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448797051 0 0) 0x7f49200035d0 con 0x7f49340d21d0 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 >> v2:172.21.15.169:3300/0 conn(0x7f49340cca20 msgr2=0x7f49340d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 --2- 172.21.15.22:0/3506897992 >> v2:172.21.15.169:3300/0 conn(0x7f49340cca20 0x7f49340d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 >> v1:172.21.15.169:6789/0 conn(0x7f49340b1920 legacy=0x7f49340cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340cf410 con 0x7f49340d21d0 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f492c0019f0 con 0x7f49340d21d0 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 >> v1:172.21.15.22:6789/0 conn(0x7f49340d21d0 legacy=0x7f49340d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4920003c00 0x7f4920003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920004870 0x7f4920004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 0x7f492000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340b1cf0 con 0x7f4920005220 2023-06-23T18:04:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492000c100 con 0x7f4920004870 2023-06-23T18:04:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492000c290 con 0x7f4920003c00 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 0x7f492000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4931d9b700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920004870 0x7f4920004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493259c700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4920003c00 0x7f4920003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 -- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4920003c00 msgr2=0x7f4920003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4920003c00 0x7f4920003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 -- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920004870 msgr2=0x7f4920004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920004870 0x7f4920004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 -- 172.21.15.22:0/3506897992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200035d0 con 0x7f4920005220 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493259c700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4920003c00 0x7f4920003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 -- 172.21.15.22:0/3506897992 reap_dead start 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f4932d9d700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 0x7f492000ba30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f492c002230 tx=0x7f49200058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab868343ea107aca server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0070d0 con 0x7f4920005220 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c007230 con 0x7f4920005220 2023-06-23T18:04:35.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493159a700 1 -- 172.21.15.22:0/3506897992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0075e0 con 0x7f4920005220 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 msgr2=0x7f492000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 0x7f492000ba30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f492c002230 tx=0x7f49200058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- 172.21.15.22:0/3506897992 shutdown_connections 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 --2- 172.21.15.22:0/3506897992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4920005220 0x7f492000ba30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- 172.21.15.22:0/3506897992 >> 172.21.15.22:0/3506897992 conn(0x7f4934069960 msgr2=0x7f4934069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- 172.21.15.22:0/3506897992 shutdown_connections 2023-06-23T18:04:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.538+0000 7f493359e700 1 -- 172.21.15.22:0/3506897992 wait complete. 2023-06-23T18:04:35.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 Processor -- start 2023-06-23T18:04:35.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 -- start start 2023-06-23T18:04:35.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920005220 0x7f49340b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49340b5c50 0x7f4934150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340b7e40 con 0x7f49341f5630 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340b60c0 con 0x7f4920005220 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f493359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49340b65d0 con 0x7f49340b5c50 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f4931d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f4931d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54634/0 (socket says 172.21.15.22:54634) 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.542+0000 7f4931d9b700 1 -- 172.21.15.22:0/2422047538 learned_addr learned my addr 172.21.15.22:0/2422047538 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:35.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f493259c700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920005220 0x7f49340b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4932d9d700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49340b5c50 0x7f4934150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 -- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49340b5c50 msgr2=0x7f4934150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49340b5c50 0x7f4934150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 -- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920005220 msgr2=0x7f49340b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920005220 0x7f49340b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 -- 172.21.15.22:0/2422047538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340b62f0 con 0x7f49341f5630 2023-06-23T18:04:35.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f4931d9b700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49240119a0 tx=0x7f4924011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400f750 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400f8d0 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49341f4b50 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400fc80 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49341f5110 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493404e4c0 con 0x7f49341f5630 2023-06-23T18:04:35.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.546+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4924014440 con 0x7f49341f5630 2023-06-23T18:04:35.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.550+0000 7f492affd700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 0x7f49140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.550+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4924084690 con 0x7f49341f5630 2023-06-23T18:04:35.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.550+0000 7f493259c700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 0x7f49140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.550+0000 7f493259c700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 0x7f49140687a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f491c007fd0 tx=0x7f491c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.554+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4924051740 con 0x7f49341f5630 2023-06-23T18:04:35.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.722+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49340c8dd0 con 0x7f49341f5630 2023-06-23T18:04:35.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.722+0000 7f492affd700 1 -- 172.21.15.22:0/2422047538 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4924054d60 con 0x7f49341f5630 2023-06-23T18:04:35.736 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 msgr2=0x7f49140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 0x7f49140687a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f491c007fd0 tx=0x7f491c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 msgr2=0x7f49340b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49240119a0 tx=0x7f4924011cb0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 shutdown_connections 2023-06-23T18:04:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49340b5c50 0x7f4934150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49341f5630 0x7f49340b5710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f49140662f0 0x7f49140687a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 --2- 172.21.15.22:0/2422047538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4920005220 0x7f49340b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 >> 172.21.15.22:0/2422047538 conn(0x7f4934069960 msgr2=0x7f49340d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 shutdown_connections 2023-06-23T18:04:35.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.726+0000 7f493359e700 1 -- 172.21.15.22:0/2422047538 wait complete. 2023-06-23T18:04:35.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.862+0000 7f2d299c8700 1 Processor -- start 2023-06-23T18:04:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.862+0000 7f2d299c8700 1 -- start start 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d240b1920 0x7f2d240cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240d1d40 con 0x7f2d240d21d0 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240cf130 con 0x7f2d240cca20 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240b1cf0 con 0x7f2d240b1920 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d291c7700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2d240d21d0 0x7f2d240d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58470/0 (socket says 172.21.15.22:58470) 2023-06-23T18:04:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d289c6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d240b1920 0x7f2d240cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d291c7700 1 -- 172.21.15.22:0/2284780444 learned_addr learned my addr 172.21.15.22:0/2284780444 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073628691 0 0) 0x7f2d240d1d40 con 0x7f2d240d21d0 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d100035d0 con 0x7f2d240d21d0 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d289c6700 1 -- 172.21.15.22:0/2284780444 >> v1:172.21.15.169:6789/0 conn(0x7f2d240cca20 legacy=0x7f2d240d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505780323 0 0) 0x7f2d100035d0 con 0x7f2d240d21d0 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d289c6700 1 -- 172.21.15.22:0/2284780444 >> v1:172.21.15.22:6789/0 conn(0x7f2d240d21d0 legacy=0x7f2d240d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d289c6700 1 -- 172.21.15.22:0/2284780444 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240cf410 con 0x7f2d240b1920 2023-06-23T18:04:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d289c6700 1 --2- 172.21.15.22:0/2284780444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240b1920 0x7f2d240cffa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009850 tx=0x7f2d0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aeb5c51787c391 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0019f0 con 0x7f2d240b1920 2023-06-23T18:04:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c010590 con 0x7f2d240b1920 2023-06-23T18:04:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d237fe700 1 -- 172.21.15.22:0/2284780444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c010940 con 0x7f2d240b1920 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- 172.21.15.22:0/2284780444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240b1920 msgr2=0x7f2d240cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 --2- 172.21.15.22:0/2284780444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240b1920 0x7f2d240cffa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009850 tx=0x7f2d0c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- 172.21.15.22:0/2284780444 shutdown_connections 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 --2- 172.21.15.22:0/2284780444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240b1920 0x7f2d240cffa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- 172.21.15.22:0/2284780444 >> 172.21.15.22:0/2284780444 conn(0x7f2d24069960 msgr2=0x7f2d24069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- 172.21.15.22:0/2284780444 shutdown_connections 2023-06-23T18:04:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.866+0000 7f2d299c8700 1 -- 172.21.15.22:0/2284780444 wait complete. 2023-06-23T18:04:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 Processor -- start 2023-06-23T18:04:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 -- start start 2023-06-23T18:04:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 0x7f2d240b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d240cca20 0x7f2d241f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b6580 con 0x7f2d240d21d0 2023-06-23T18:04:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b60f0 con 0x7f2d240b1920 2023-06-23T18:04:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d299c8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b5200 con 0x7f2d240cca20 2023-06-23T18:04:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54646/0 (socket says 172.21.15.22:54646) 2023-06-23T18:04:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 -- 172.21.15.22:0/428318274 learned_addr learned my addr 172.21.15.22:0/428318274 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d289c6700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 0x7f2d240b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d23fff700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d240cca20 0x7f2d241f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 -- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d240cca20 msgr2=0x7f2d241f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d240cca20 0x7f2d241f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 -- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 msgr2=0x7f2d240b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 0x7f2d240b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 -- 172.21.15.22:0/428318274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0094b0 con 0x7f2d240d21d0 2023-06-23T18:04:35.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d289c6700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 0x7f2d240b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:35.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d291c7700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00cbe0 tx=0x7f2d1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.870+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00abc0 con 0x7f2d240d21d0 2023-06-23T18:04:35.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c00ad20 con 0x7f2d240d21d0 2023-06-23T18:04:35.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d240b54e0 con 0x7f2d240d21d0 2023-06-23T18:04:35.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c007310 con 0x7f2d240d21d0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d240b5aa0 con 0x7f2d240d21d0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2404e4c0 con 0x7f2d240d21d0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2d1c007470 con 0x7f2d240d21d0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d217fa700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 0x7f2d040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d289c6700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 0x7f2d040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.874+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2d1c083f70 con 0x7f2d240d21d0 2023-06-23T18:04:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.878+0000 7f2d289c6700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 0x7f2d040687a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c013040 tx=0x7f2d0c008000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:35.882+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2d1c0510b0 con 0x7f2d240d21d0 2023-06-23T18:04:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.046+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d240b46d0 con 0x7f2d240d21d0 2023-06-23T18:04:36.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.046+0000 7f2d217fa700 1 -- 172.21.15.22:0/428318274 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d1c0180a0 con 0x7f2d240d21d0 2023-06-23T18:04:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.046+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 msgr2=0x7f2d040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 0x7f2d040687a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c013040 tx=0x7f2d0c008000 comp rx=0 tx=0).stop 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 msgr2=0x7f2d241504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00cbe0 tx=0x7f2d1c00a060 comp rx=0 tx=0).stop 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 shutdown_connections 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d240d21d0 0x7f2d241504b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d240cca20 0x7f2d241f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2d040662f0 0x7f2d040687a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 --2- 172.21.15.22:0/428318274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d240b1920 0x7f2d240b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 >> 172.21.15.22:0/428318274 conn(0x7f2d24069960 msgr2=0x7f2d240c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 shutdown_connections 2023-06-23T18:04:36.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.050+0000 7f2d299c8700 1 -- 172.21.15.22:0/428318274 wait complete. 2023-06-23T18:04:36.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.182+0000 7f6c2b47a700 1 Processor -- start 2023-06-23T18:04:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.182+0000 7f6c2b47a700 1 -- start start 2023-06-23T18:04:36.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c24159b40 0x7f6c240af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c240b02a0 0x7f6c240b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2406fe20 con 0x7f6c2415e1b0 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240b0be0 con 0x7f6c24159b40 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2415e600 con 0x7f6c240b02a0 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29c77700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6c2415e1b0 0x7f6c24069960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58482/0 (socket says 172.21.15.22:58482) 2023-06-23T18:04:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29c77700 1 -- 172.21.15.22:0/109978217 learned_addr learned my addr 172.21.15.22:0/109978217 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 --2- 172.21.15.22:0/109978217 >> v2:172.21.15.22:3300/0 conn(0x7f6c24159b40 0x7f6c240af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2ac79700 1 --2- 172.21.15.22:0/109978217 >> v2:172.21.15.169:3300/0 conn(0x7f6c240b02a0 0x7f6c240b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283098676 0 0) 0x7f6c2406fe20 con 0x7f6c2415e1b0 2023-06-23T18:04:36.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c100035d0 con 0x7f6c2415e1b0 2023-06-23T18:04:36.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 -- 172.21.15.22:0/109978217 >> v2:172.21.15.169:3300/0 conn(0x7f6c240b02a0 msgr2=0x7f6c240b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:36.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661377408 0 0) 0x7f6c100035d0 con 0x7f6c2415e1b0 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 --2- 172.21.15.22:0/109978217 >> v2:172.21.15.169:3300/0 conn(0x7f6c240b02a0 0x7f6c240b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 -- 172.21.15.22:0/109978217 >> v1:172.21.15.22:6789/0 conn(0x7f6c2415e1b0 legacy=0x7f6c24069960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 -- 172.21.15.22:0/109978217 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240aff00 con 0x7f6c24159b40 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2a478700 1 --2- 172.21.15.22:0/109978217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240af800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c2406b420 tx=0x7f6c20009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6208b8eccd2826aa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14001670 con 0x7f6c24159b40 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20004030 con 0x7f6c24159b40 2023-06-23T18:04:36.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c29476700 1 -- 172.21.15.22:0/109978217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200043c0 con 0x7f6c24159b40 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- 172.21.15.22:0/109978217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 msgr2=0x7f6c240af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/109978217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240af800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c2406b420 tx=0x7f6c20009b30 comp rx=0 tx=0).stop 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- 172.21.15.22:0/109978217 shutdown_connections 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/109978217 >> v2:172.21.15.169:3300/0 conn(0x7f6c240b02a0 0x7f6c240b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/109978217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240af800 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- 172.21.15.22:0/109978217 >> 172.21.15.22:0/109978217 conn(0x7f6c2406ca60 msgr2=0x7f6c240ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- 172.21.15.22:0/109978217 shutdown_connections 2023-06-23T18:04:36.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.186+0000 7f6c2b47a700 1 -- 172.21.15.22:0/109978217 wait complete. 2023-06-23T18:04:36.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 Processor -- start 2023-06-23T18:04:36.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 -- start start 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 0x7f6c2414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2414fcf0 0x7f6c240aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24151f50 con 0x7f6c24159b40 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cd410 con 0x7f6c2414fcf0 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2b47a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cd6f0 con 0x7f6c2415e1b0 2023-06-23T18:04:36.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2a478700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2a478700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54664/0 (socket says 172.21.15.22:54664) 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2a478700 1 -- 172.21.15.22:0/832933199 learned_addr learned my addr 172.21.15.22:0/832933199 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c29c77700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 0x7f6c2414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2ac79700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2414fcf0 0x7f6c240aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.190+0000 7f6c2a478700 1 -- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 msgr2=0x7f6c2414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2a478700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 0x7f6c2414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2a478700 1 -- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2414fcf0 msgr2=0x7f6c240aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2a478700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2414fcf0 0x7f6c240aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2a478700 1 -- 172.21.15.22:0/832933199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200094b0 con 0x7f6c24159b40 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c29c77700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 0x7f6c2414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2a478700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6c20000c00 tx=0x7f6c20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003a40 con 0x7f6c24159b40 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2001f070 con 0x7f6c24159b40 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240cd9d0 con 0x7f6c24159b40 2023-06-23T18:04:36.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003dc0 con 0x7f6c24159b40 2023-06-23T18:04:36.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c24150190 con 0x7f6c24159b40 2023-06-23T18:04:36.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2404e4c0 con 0x7f6c24159b40 2023-06-23T18:04:36.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6c20021030 con 0x7f6c24159b40 2023-06-23T18:04:36.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.194+0000 7f6c1affd700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 0x7f6c080687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.198+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c20085670 con 0x7f6c24159b40 2023-06-23T18:04:36.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.198+0000 7f6c29c77700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 0x7f6c080687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.198+0000 7f6c29c77700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 0x7f6c080687a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14001230 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.202+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6c20052850 con 0x7f6c24159b40 2023-06-23T18:04:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6c240b11e0 con 0x7f6c24159b40 2023-06-23T18:04:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c1affd700 1 -- 172.21.15.22:0/832933199 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6c20055e70 con 0x7f6c24159b40 2023-06-23T18:04:36.385 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:36.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 msgr2=0x7f6c080687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 0x7f6c080687a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14001230 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 msgr2=0x7f6c240ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.370+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6c20000c00 tx=0x7f6c20004770 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 shutdown_connections 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6c080662f0 0x7f6c080687a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2414fcf0 0x7f6c240aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2415e1b0 0x7f6c2414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 --2- 172.21.15.22:0/832933199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c24159b40 0x7f6c240ccde0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 >> 172.21.15.22:0/832933199 conn(0x7f6c240b02a0 msgr2=0x7f6c240ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 shutdown_connections 2023-06-23T18:04:36.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.374+0000 7f6c2b47a700 1 -- 172.21.15.22:0/832933199 wait complete. 2023-06-23T18:04:36.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:36.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.494+0000 7fe62936f700 1 Processor -- start 2023-06-23T18:04:36.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- start start 2023-06-23T18:04:36.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 0x7fe624069960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe624069ea0 0x7fe62415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62415c960 con 0x7fe6240ccb00 2023-06-23T18:04:36.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62415cc40 con 0x7fe624069ea0 2023-06-23T18:04:36.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62415cf20 con 0x7fe62415c3e0 2023-06-23T18:04:36.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe624069ea0 0x7fe62415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe622ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe6240ccb00 0x7fe6240cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59076/0 (socket says 172.21.15.22:59076) 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe622ffd700 1 -- 172.21.15.22:0/3628685020 learned_addr learned my addr 172.21.15.22:0/3628685020 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6227fc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 0x7fe624069960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 -- 172.21.15.22:0/3628685020 >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 msgr2=0x7fe624069960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 --2- 172.21.15.22:0/3628685020 >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 0x7fe624069960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 -- 172.21.15.22:0/3628685020 >> v1:172.21.15.169:6789/0 conn(0x7fe6240ccb00 legacy=0x7fe6240cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 -- 172.21.15.22:0/3628685020 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62415d200 con 0x7fe624069ea0 2023-06-23T18:04:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6227fc700 1 --2- 172.21.15.22:0/3628685020 >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 0x7fe624069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe6237fe700 1 --2- 172.21.15.22:0/3628685020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe624069ea0 0x7fe62415fba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe62415fae0 tx=0x7fe614009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c0df2399f16ede server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe621ffb700 1 -- 172.21.15.22:0/3628685020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614011830 con 0x7fe624069ea0 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe621ffb700 1 -- 172.21.15.22:0/3628685020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe614011990 con 0x7fe624069ea0 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe621ffb700 1 -- 172.21.15.22:0/3628685020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614011d20 con 0x7fe624069ea0 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- 172.21.15.22:0/3628685020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe624069ea0 msgr2=0x7fe62415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 --2- 172.21.15.22:0/3628685020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe624069ea0 0x7fe62415fba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe62415fae0 tx=0x7fe614009b30 comp rx=0 tx=0).stop 2023-06-23T18:04:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- 172.21.15.22:0/3628685020 shutdown_connections 2023-06-23T18:04:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 --2- 172.21.15.22:0/3628685020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe624069ea0 0x7fe62415fba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 --2- 172.21.15.22:0/3628685020 >> v2:172.21.15.169:3300/0 conn(0x7fe62415c3e0 0x7fe624069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- 172.21.15.22:0/3628685020 >> 172.21.15.22:0/3628685020 conn(0x7fe62406c080 msgr2=0x7fe6240d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- 172.21.15.22:0/3628685020 shutdown_connections 2023-06-23T18:04:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.498+0000 7fe62936f700 1 -- 172.21.15.22:0/3628685020 wait complete. 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 Processor -- start 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- start start 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe624069ea0 0x7fe6240c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6240ccb00 0x7fe6240b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240c6b50 con 0x7fe62415c3e0 2023-06-23T18:04:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240c6e30 con 0x7fe6240ccb00 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240c7110 con 0x7fe624069ea0 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46396/0 (socket says 172.21.15.22:46396) 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 -- 172.21.15.22:0/605985791 learned_addr learned my addr 172.21.15.22:0/605985791 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe622ffd700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe624069ea0 0x7fe6240c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6227fc700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6240ccb00 0x7fe6240b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 -- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe624069ea0 msgr2=0x7fe6240c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe624069ea0 0x7fe6240c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 -- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6240ccb00 msgr2=0x7fe6240b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6240ccb00 0x7fe6240b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 -- 172.21.15.22:0/605985791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0094b0 con 0x7fe62415c3e0 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe6237fe700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140044d0 con 0x7fe62415c3e0 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61401e070 con 0x7fe62415c3e0 2023-06-23T18:04:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140094b0 con 0x7fe62415c3e0 2023-06-23T18:04:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614003900 con 0x7fe62415c3e0 2023-06-23T18:04:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.502+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6240c76d0 con 0x7fe62415c3e0 2023-06-23T18:04:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62404dc70 con 0x7fe62415c3e0 2023-06-23T18:04:36.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe614003a60 con 0x7fe62415c3e0 2023-06-23T18:04:36.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe61b7fe700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 0x7fe6040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe614084d70 con 0x7fe62415c3e0 2023-06-23T18:04:36.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe622ffd700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 0x7fe6040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.506+0000 7fe622ffd700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 0x7fe6040687a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0012e0 tx=0x7fe61c001990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.514+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe614051ed0 con 0x7fe62415c3e0 2023-06-23T18:04:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.690+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe6240c5d80 con 0x7fe62415c3e0 2023-06-23T18:04:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.690+0000 7fe61b7fe700 1 -- 172.21.15.22:0/605985791 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe6140554f0 con 0x7fe62415c3e0 2023-06-23T18:04:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 msgr2=0x7fe6040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 0x7fe6040687a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0012e0 tx=0x7fe61c001990 comp rx=0 tx=0).stop 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 msgr2=0x7fe6240c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614004770 comp rx=0 tx=0).stop 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 shutdown_connections 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62415c3e0 0x7fe6240c5fe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6040662f0 0x7fe6040687a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6240ccb00 0x7fe6240b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 --2- 172.21.15.22:0/605985791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe624069ea0 0x7fe6240c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 >> 172.21.15.22:0/605985791 conn(0x7fe62406c080 msgr2=0x7fe6240b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 shutdown_connections 2023-06-23T18:04:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.694+0000 7fe62936f700 1 -- 172.21.15.22:0/605985791 wait complete. 2023-06-23T18:04:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:36.726 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:36.726 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:36.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 Processor -- start 2023-06-23T18:04:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 -- start start 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68300ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6830156540 0x7f68300ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6830160140 con 0x7f68300710a0 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683015f850 con 0x7f6830156540 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f6836ada700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300cee40 con 0x7f68301605d0 2023-06-23T18:04:36.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.842+0000 7f68362d9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6830156540 0x7f68300ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6830156540 0x7f68300ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46404/0 (socket says 172.21.15.22:46404) 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 -- 172.21.15.22:0/1495266976 learned_addr learned my addr 172.21.15.22:0/1495266976 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68352d7700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68300ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 -- 172.21.15.22:0/1495266976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 msgr2=0x7f68300ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68300ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 -- 172.21.15.22:0/1495266976 >> v1:172.21.15.169:6789/0 conn(0x7f68300710a0 legacy=0x7f6830071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 -- 172.21.15.22:0/1495266976 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683015faf0 con 0x7f6830156540 2023-06-23T18:04:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68352d7700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68300ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:36.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f68362d9700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6830156540 0x7f68300ce7c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6828009910 tx=0x7f6828009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfddde217ed02c2e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6834ad6700 1 -- 172.21.15.22:0/1495266976 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800ba60 con 0x7f6830156540 2023-06-23T18:04:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6834ad6700 1 -- 172.21.15.22:0/1495266976 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800bbc0 con 0x7f6830156540 2023-06-23T18:04:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6834ad6700 1 -- 172.21.15.22:0/1495266976 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009e60 con 0x7f6830156540 2023-06-23T18:04:36.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- 172.21.15.22:0/1495266976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6830156540 msgr2=0x7f68300ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6830156540 0x7f68300ce7c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6828009910 tx=0x7f6828009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- 172.21.15.22:0/1495266976 shutdown_connections 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6830156540 0x7f68300ce7c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 --2- 172.21.15.22:0/1495266976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68300ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- 172.21.15.22:0/1495266976 >> 172.21.15.22:0/1495266976 conn(0x7f683006aa70 msgr2=0x7f683006ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- 172.21.15.22:0/1495266976 shutdown_connections 2023-06-23T18:04:36.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- 172.21.15.22:0/1495266976 wait complete. 2023-06-23T18:04:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 Processor -- start 2023-06-23T18:04:36.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.846+0000 7f6836ada700 1 -- start start 2023-06-23T18:04:36.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68301e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68301e7a60 0x7f68301f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301e7100 con 0x7f68300710a0 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301e6db0 con 0x7f68301e7a60 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68301f1de0 con 0x7f68301605d0 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46420/0 (socket says 172.21.15.22:46420) 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 -- 172.21.15.22:0/1266153149 learned_addr learned my addr 172.21.15.22:0/1266153149 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f68352d7700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68301e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f68362d9700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68301e7a60 0x7f68301f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 -- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 msgr2=0x7f68301e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68301e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 -- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68301e7a60 msgr2=0x7f68301f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68301e7a60 0x7f68301f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 -- 172.21.15.22:0/1266153149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009570 con 0x7f68300710a0 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6835ad8700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c530 tx=0x7f682c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0052d0 con 0x7f68300710a0 2023-06-23T18:04:36.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c005430 con 0x7f68300710a0 2023-06-23T18:04:36.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301f20c0 con 0x7f68300710a0 2023-06-23T18:04:36.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0055a0 con 0x7f68300710a0 2023-06-23T18:04:36.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.850+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301f2c60 con 0x7f68300710a0 2023-06-23T18:04:36.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683004e4c0 con 0x7f68300710a0 2023-06-23T18:04:36.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f682c0038e0 con 0x7f68300710a0 2023-06-23T18:04:36.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f68227fc700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 0x7f68140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:36.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f682c083470 con 0x7f68300710a0 2023-06-23T18:04:36.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f68352d7700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 0x7f68140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:36.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.854+0000 7f68352d7700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 0x7f68140687a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f683015bb20 tx=0x7f68240097f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:36.862+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f682c050520 con 0x7f68300710a0 2023-06-23T18:04:37.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.026+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68300b50e0 con 0x7f68300710a0 2023-06-23T18:04:37.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.030+0000 7f68227fc700 1 -- 172.21.15.22:0/1266153149 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f682c053b40 con 0x7f68300710a0 2023-06-23T18:04:37.044 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:37.045 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:37.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 msgr2=0x7f68140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 0x7f68140687a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f683015bb20 tx=0x7f68240097f0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 msgr2=0x7f683015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c530 tx=0x7f682c00c840 comp rx=0 tx=0).stop 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 shutdown_connections 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68140662f0 0x7f68140687a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68301e7a60 0x7f68301f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68301605d0 0x7f68301e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 --2- 172.21.15.22:0/1266153149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68300710a0 0x7f683015de30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 >> 172.21.15.22:0/1266153149 conn(0x7f683006aa70 msgr2=0x7f68300ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 shutdown_connections 2023-06-23T18:04:37.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.034+0000 7f6836ada700 1 -- 172.21.15.22:0/1266153149 wait complete. 2023-06-23T18:04:37.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:37.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.166+0000 7f7286fd1700 1 Processor -- start 2023-06-23T18:04:37.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.166+0000 7f7286fd1700 1 -- start start 2023-06-23T18:04:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72800d1f30 0x7f72800d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 0x7f72800d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800ccb30 con 0x7f72800cf3a0 2023-06-23T18:04:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72800d2880 con 0x7f72800d1f30 2023-06-23T18:04:37.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800d2b60 con 0x7f72800d2360 2023-06-23T18:04:37.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7285fcf700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f72800cf3a0 0x7f72800cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51652/0 (socket says 172.21.15.22:51652) 2023-06-23T18:04:37.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7285fcf700 1 -- 172.21.15.22:0/3902108314 learned_addr learned my addr 172.21.15.22:0/3902108314 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:37.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 --2- 172.21.15.22:0/3902108314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 0x7f72800d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72857ce700 1 --2- 172.21.15.22:0/3902108314 >> v2:172.21.15.169:3300/0 conn(0x7f72800d1f30 0x7f72800d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7284fcd700 1 -- 172.21.15.22:0/3902108314 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917355455 0 0) 0x7f72800ccb30 con 0x7f72800cf3a0 2023-06-23T18:04:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7284fcd700 1 -- 172.21.15.22:0/3902108314 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f726c0035d0 con 0x7f72800cf3a0 2023-06-23T18:04:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 -- 172.21.15.22:0/3902108314 >> v2:172.21.15.169:3300/0 conn(0x7f72800d1f30 msgr2=0x7f72800d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 --2- 172.21.15.22:0/3902108314 >> v2:172.21.15.169:3300/0 conn(0x7f72800d1f30 0x7f72800d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 -- 172.21.15.22:0/3902108314 >> v1:172.21.15.22:6789/0 conn(0x7f72800cf3a0 legacy=0x7f72800cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 -- 172.21.15.22:0/3902108314 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72800d2e40 con 0x7f72800d2360 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f72867d0700 1 --2- 172.21.15.22:0/3902108314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 0x7f72800d0560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7278009d90 tx=0x7f727800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962d0418f21e5fd0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7284fcd700 1 -- 172.21.15.22:0/3902108314 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800e070 con 0x7f72800d2360 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7284fcd700 1 -- 172.21.15.22:0/3902108314 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800bb20 con 0x7f72800d2360 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7284fcd700 1 -- 172.21.15.22:0/3902108314 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800be90 con 0x7f72800d2360 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- 172.21.15.22:0/3902108314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 msgr2=0x7f72800d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 --2- 172.21.15.22:0/3902108314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 0x7f72800d0560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7278009d90 tx=0x7f727800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- 172.21.15.22:0/3902108314 shutdown_connections 2023-06-23T18:04:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 --2- 172.21.15.22:0/3902108314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800d2360 0x7f72800d0560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 --2- 172.21.15.22:0/3902108314 >> v2:172.21.15.169:3300/0 conn(0x7f72800d1f30 0x7f72800d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- 172.21.15.22:0/3902108314 >> 172.21.15.22:0/3902108314 conn(0x7f728006a280 msgr2=0x7f728009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- 172.21.15.22:0/3902108314 shutdown_connections 2023-06-23T18:04:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.170+0000 7f7286fd1700 1 -- 172.21.15.22:0/3902108314 wait complete. 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 Processor -- start 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 -- start start 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 0x7f7280159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 0x7f72801f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728015c180 con 0x7f72800d2360 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728015c3f0 con 0x7f72800d1f30 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7286fd1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728015c6d0 con 0x7f72800cf3a0 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f72867d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f72867d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46432/0 (socket says 172.21.15.22:46432) 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f72867d0700 1 -- 172.21.15.22:0/1659197456 learned_addr learned my addr 172.21.15.22:0/1659197456 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f72857ce700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 0x7f72801f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.174+0000 7f7285fcf700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 0x7f7280159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 -- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 msgr2=0x7f7280159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 0x7f7280159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 -- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 msgr2=0x7f72801f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 0x7f72801f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 -- 172.21.15.22:0/1659197456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278009a70 con 0x7f72800d2360 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f7285fcf700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 0x7f7280159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72857ce700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 0x7f72801f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72867d0700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7278014040 tx=0x7f7278004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278007000 con 0x7f72800d2360 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278012860 con 0x7f72800d2360 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728015c9b0 con 0x7f72800d2360 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278012c10 con 0x7f72800d2360 2023-06-23T18:04:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72801f2d30 con 0x7f72800d2360 2023-06-23T18:04:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728015ced0 con 0x7f72800d2360 2023-06-23T18:04:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f727801c400 con 0x7f72800d2360 2023-06-23T18:04:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 0x7f72640687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7278085ec0 con 0x7f72800d2360 2023-06-23T18:04:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.178+0000 7f7285fcf700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 0x7f72640687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.182+0000 7f7285fcf700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 0x7f72640687d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f727c001360 tx=0x7f727c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.190+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f728015ced0 con 0x7f72800d2360 2023-06-23T18:04:37.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.354+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f728015ced0 con 0x7f72800d2360 2023-06-23T18:04:37.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.354+0000 7f72727fc700 1 -- 172.21.15.22:0/1659197456 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f728015ced0 con 0x7f72800d2360 2023-06-23T18:04:37.369 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:37.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 msgr2=0x7f72640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 0x7f72640687d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f727c001360 tx=0x7f727c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 msgr2=0x7f728006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7278014040 tx=0x7f7278004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 shutdown_connections 2023-06-23T18:04:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72800d2360 0x7f728006b7c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7264066320 0x7f72640687d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72800d1f30 0x7f72801f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 --2- 172.21.15.22:0/1659197456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72800cf3a0 0x7f7280159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 >> 172.21.15.22:0/1659197456 conn(0x7f728006a280 msgr2=0x7f728015b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 shutdown_connections 2023-06-23T18:04:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.358+0000 7f7286fd1700 1 -- 172.21.15.22:0/1659197456 wait complete. 2023-06-23T18:04:37.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:37.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.486+0000 7f8b77fff700 1 Processor -- start 2023-06-23T18:04:37.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.486+0000 7f8b77fff700 1 -- start start 2023-06-23T18:04:37.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 0x7f8b7806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b78158ef0 0x7f8b780b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7806eab0 con 0x7f8b78070740 2023-06-23T18:04:37.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b780cdf20 con 0x7f8b78158ef0 2023-06-23T18:04:37.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780cd680 con 0x7f8b78158b20 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b6e7fc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b78158ef0 0x7f8b780b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 0x7f8b7806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b777fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8b78070740 0x7f8b7815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59084/0 (socket says 172.21.15.22:59084) 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b777fe700 1 -- 172.21.15.22:0/3479228289 learned_addr learned my addr 172.21.15.22:0/3479228289 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 -- 172.21.15.22:0/3479228289 >> v2:172.21.15.169:3300/0 conn(0x7f8b78158ef0 msgr2=0x7f8b780b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 --2- 172.21.15.22:0/3479228289 >> v2:172.21.15.169:3300/0 conn(0x7f8b78158ef0 0x7f8b780b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 -- 172.21.15.22:0/3479228289 >> v1:172.21.15.169:6789/0 conn(0x7f8b78070740 legacy=0x7f8b7815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 -- 172.21.15.22:0/3479228289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780cd960 con 0x7f8b78158b20 2023-06-23T18:04:37.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b767fc700 1 -- 172.21.15.22:0/3479228289 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878276591 0 0) 0x7f8b7806eab0 con 0x7f8b78070740 2023-06-23T18:04:37.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b76ffd700 1 --2- 172.21.15.22:0/3479228289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 0x7f8b7806e440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009850 tx=0x7f8b68009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d3bfa08247ed9d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b767fc700 1 -- 172.21.15.22:0/3479228289 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011830 con 0x7f8b78158b20 2023-06-23T18:04:37.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b767fc700 1 -- 172.21.15.22:0/3479228289 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68011990 con 0x7f8b78158b20 2023-06-23T18:04:37.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b767fc700 1 -- 172.21.15.22:0/3479228289 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011d40 con 0x7f8b78158b20 2023-06-23T18:04:37.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- 172.21.15.22:0/3479228289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 msgr2=0x7f8b7806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 --2- 172.21.15.22:0/3479228289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 0x7f8b7806e440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009850 tx=0x7f8b68009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- 172.21.15.22:0/3479228289 shutdown_connections 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 --2- 172.21.15.22:0/3479228289 >> v2:172.21.15.169:3300/0 conn(0x7f8b78158ef0 0x7f8b780b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 --2- 172.21.15.22:0/3479228289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158b20 0x7f8b7806e440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- 172.21.15.22:0/3479228289 >> 172.21.15.22:0/3479228289 conn(0x7f8b7806b700 msgr2=0x7f8b780b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- 172.21.15.22:0/3479228289 shutdown_connections 2023-06-23T18:04:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.490+0000 7f8b77fff700 1 -- 172.21.15.22:0/3479228289 wait complete. 2023-06-23T18:04:37.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 Processor -- start 2023-06-23T18:04:37.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 -- start start 2023-06-23T18:04:37.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 0x7f8b7815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 0x7f8b7815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78143d20 con 0x7f8b78070740 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78145aa0 con 0x7f8b78158b20 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b77fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78159ed0 con 0x7f8b78158ef0 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46440/0 (socket says 172.21.15.22:46440) 2023-06-23T18:04:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 -- 172.21.15.22:0/451474131 learned_addr learned my addr 172.21.15.22:0/451474131 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:37.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b777fe700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 0x7f8b7815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b6e7fc700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 0x7f8b7815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 -- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 msgr2=0x7f8b7815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 0x7f8b7815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 -- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 msgr2=0x7f8b7815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 0x7f8b7815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 -- 172.21.15.22:0/451474131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680094b0 con 0x7f8b78070740 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b777fe700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 0x7f8b7815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b6e7fc700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 0x7f8b7815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:37.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.494+0000 7f8b76ffd700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000c00 tx=0x7f8b68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6801aac0 con 0x7f8b78070740 2023-06-23T18:04:37.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7815a1b0 con 0x7f8b78070740 2023-06-23T18:04:37.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b78143940 con 0x7f8b78070740 2023-06-23T18:04:37.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6801f070 con 0x7f8b78070740 2023-06-23T18:04:37.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6801ae20 con 0x7f8b78070740 2023-06-23T18:04:37.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7804e4c0 con 0x7f8b78070740 2023-06-23T18:04:37.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.498+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8b6801b430 con 0x7f8b78070740 2023-06-23T18:04:37.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.502+0000 7f8b6ffff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 0x7f8b580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.502+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8b68085220 con 0x7f8b78070740 2023-06-23T18:04:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.502+0000 7f8b6e7fc700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 0x7f8b580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.502+0000 7f8b6e7fc700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 0x7f8b580687a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b78143530 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.510+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8b680522e0 con 0x7f8b78070740 2023-06-23T18:04:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.674+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b780b1560 con 0x7f8b78070740 2023-06-23T18:04:37.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.674+0000 7f8b6ffff700 1 -- 172.21.15.22:0/451474131 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b68055900 con 0x7f8b78070740 2023-06-23T18:04:37.689 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 msgr2=0x7f8b580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 0x7f8b580687a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b78143530 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 msgr2=0x7f8b780afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000c00 tx=0x7f8b68004770 comp rx=0 tx=0).stop 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 shutdown_connections 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b78158ef0 0x7f8b7815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f8b580662f0 0x7f8b580687a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b78158b20 0x7f8b7815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 --2- 172.21.15.22:0/451474131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b78070740 0x7f8b780afcd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 >> 172.21.15.22:0/451474131 conn(0x7f8b7806b700 msgr2=0x7f8b780c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 shutdown_connections 2023-06-23T18:04:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.678+0000 7f8b77fff700 1 -- 172.21.15.22:0/451474131 wait complete. 2023-06-23T18:04:37.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:37.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 Processor -- start 2023-06-23T18:04:37.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 -- start start 2023-06-23T18:04:37.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f64740d0730 0x7f64740ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64740ccb30 con 0x7f64740d1f30 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64740cfcb0 con 0x7f64740d0730 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64740d2880 con 0x7f64740cf3a0 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f64740d0730 0x7f64740ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f64740d0730 0x7f64740ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46446/0 (socket says 172.21.15.22:46446) 2023-06-23T18:04:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 -- 172.21.15.22:0/3419778648 learned_addr learned my addr 172.21.15.22:0/3419778648 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6478f67700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 -- 172.21.15.22:0/3419778648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 msgr2=0x7f64740cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 -- 172.21.15.22:0/3419778648 >> v1:172.21.15.169:6789/0 conn(0x7f64740d1f30 legacy=0x7f64740d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 -- 172.21.15.22:0/3419778648 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64740d2b60 con 0x7f64740d0730 2023-06-23T18:04:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479768700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740ccfc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f646c009e50 tx=0x7f646c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca99624b0dcd259 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f64737fe700 1 -- 172.21.15.22:0/3419778648 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c00e070 con 0x7f64740d0730 2023-06-23T18:04:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f64737fe700 1 -- 172.21.15.22:0/3419778648 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c00b9c0 con 0x7f64740d0730 2023-06-23T18:04:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f64737fe700 1 -- 172.21.15.22:0/3419778648 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c00bd70 con 0x7f64740d0730 2023-06-23T18:04:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.794+0000 7f6479f69700 1 -- 172.21.15.22:0/3419778648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 msgr2=0x7f64740ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740ccfc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f646c009e50 tx=0x7f646c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/3419778648 shutdown_connections 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740ccfc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- 172.21.15.22:0/3419778648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/3419778648 >> 172.21.15.22:0/3419778648 conn(0x7f647406a280 msgr2=0x7f647409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/3419778648 shutdown_connections 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/3419778648 wait complete. 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 Processor -- start 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- start start 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 0x7f647415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64740c8820 con 0x7f64740d0730 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741f14f0 con 0x7f64740d1f30 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64741f1d70 con 0x7f64740cf3a0 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46458/0 (socket says 172.21.15.22:46458) 2023-06-23T18:04:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 -- 172.21.15.22:0/242434898 learned_addr learned my addr 172.21.15.22:0/242434898 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479768700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 0x7f647415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6478f67700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 -- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 msgr2=0x7f64740cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 -- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 msgr2=0x7f647415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 0x7f647415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 -- 172.21.15.22:0/242434898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c009b30 con 0x7f64740d0730 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479768700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 0x7f647415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6478f67700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6473fff700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f646000c500 tx=0x7f646000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64600052d0 con 0x7f64740d0730 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6460005430 con 0x7f64740d0730 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647415d2b0 con 0x7f64740d0730 2023-06-23T18:04:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64600055a0 con 0x7f64740d0730 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.798+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647415cfd0 con 0x7f64740d0730 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647404e4c0 con 0x7f64740d0730 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f64600038e0 con 0x7f64740d0730 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f64717fa700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 0x7f64580687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f64600832f0 con 0x7f64740d0730 2023-06-23T18:04:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f6478f67700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 0x7f64580687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.802+0000 7f6478f67700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 0x7f64580687d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.806+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6460050440 con 0x7f64740d0730 2023-06-23T18:04:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.966+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64741f1eb0 con 0x7f64740d0730 2023-06-23T18:04:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f64717fa700 1 -- 172.21.15.22:0/242434898 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6460053a60 con 0x7f64740d0730 2023-06-23T18:04:37.982 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:37.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 msgr2=0x7f64580687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 0x7f64580687d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6468006fd0 tx=0x7f6468008040 comp rx=0 tx=0).stop 2023-06-23T18:04:37.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 msgr2=0x7f64740c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f646000c500 tx=0x7f646000c810 comp rx=0 tx=0).stop 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 shutdown_connections 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6458066320 0x7f64580687d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64740d1f30 0x7f647415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64740d0730 0x7f64740c82e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 --2- 172.21.15.22:0/242434898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64740cf3a0 0x7f64740cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:37.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 >> 172.21.15.22:0/242434898 conn(0x7f647406a280 msgr2=0x7f647406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:37.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 shutdown_connections 2023-06-23T18:04:37.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:37.970+0000 7f6479f69700 1 -- 172.21.15.22:0/242434898 wait complete. 2023-06-23T18:04:37.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 Processor -- start 2023-06-23T18:04:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 -- start start 2023-06-23T18:04:38.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff1000cf3a0 0x7ff1000cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1000ccb30 con 0x7ff1000d2360 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000cfcb0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.106+0000 7ff1058fe700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000d2880 con 0x7ff1000d1f30 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff1000cf3a0 0x7ff1000cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1050fd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff1000d2360 0x7ff10015f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51654/0 (socket says 172.21.15.22:51654) 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1050fd700 1 -- 172.21.15.22:0/2996795462 learned_addr learned my addr 172.21.15.22:0/2996795462 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff0fffff700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 -- 172.21.15.22:0/2996795462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 msgr2=0x7ff1000d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 -- 172.21.15.22:0/2996795462 >> v1:172.21.15.22:6789/0 conn(0x7ff1000d2360 legacy=0x7ff10015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 -- 172.21.15.22:0/2996795462 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000d2b60 con 0x7ff1000cf3a0 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff0ff7fe700 1 -- 172.21.15.22:0/2996795462 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141879515 0 0) 0x7ff1000ccb30 con 0x7ff1000d2360 2023-06-23T18:04:38.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1048fc700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000cf3a0 0x7ff1000cf770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009840 tx=0x7ff0f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b383dc7d5df4ba1b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff0ff7fe700 1 -- 172.21.15.22:0/2996795462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4005e50 con 0x7ff1000cf3a0 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff0ff7fe700 1 -- 172.21.15.22:0/2996795462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4006030 con 0x7ff1000cf3a0 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff0ff7fe700 1 -- 172.21.15.22:0/2996795462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40063c0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 -- 172.21.15.22:0/2996795462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000cf3a0 msgr2=0x7ff1000cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000cf3a0 0x7ff1000cf770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009840 tx=0x7ff0f4009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 -- 172.21.15.22:0/2996795462 shutdown_connections 2023-06-23T18:04:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2996795462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000cf3a0 0x7ff1000cf770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 -- 172.21.15.22:0/2996795462 >> 172.21.15.22:0/2996795462 conn(0x7ff10006a280 msgr2=0x7ff10009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 -- 172.21.15.22:0/2996795462 shutdown_connections 2023-06-23T18:04:38.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.110+0000 7ff1058fe700 1 -- 172.21.15.22:0/2996795462 wait complete. 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 Processor -- start 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- start start 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 0x7ff1000b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000b5280 con 0x7ff1000d2360 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015ee90 con 0x7ff1000cf3a0 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000b54d0 con 0x7ff1000d1f30 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1050fd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 0x7ff1000b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1050fd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46482/0 (socket says 172.21.15.22:46482) 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1050fd700 1 -- 172.21.15.22:0/2755315198 learned_addr learned my addr 172.21.15.22:0/2755315198 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff0fffff700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 -- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 msgr2=0x7ff1000b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 -- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 msgr2=0x7ff100159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 -- 172.21.15.22:0/2755315198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009450 con 0x7ff1000cf3a0 2023-06-23T18:04:38.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1050fd700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:38.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1048fc700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 0x7ff1000b5fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4015000 tx=0x7ff0f4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40038c0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4003a40 con 0x7ff1000cf3a0 2023-06-23T18:04:38.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1000b57b0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003df0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.114+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1000b64f0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1000b5cd0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff0f4022070 con 0x7ff1000cf3a0 2023-06-23T18:04:38.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff0fd7fa700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 0x7ff0ec0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff0f4085a70 con 0x7ff1000cf3a0 2023-06-23T18:04:38.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff0fffff700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 0x7ff0ec0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.118+0000 7ff0fffff700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 0x7ff0ec0687d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1000c9640 tx=0x7ff0f0009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.126+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff0f4052b20 con 0x7ff1000cf3a0 2023-06-23T18:04:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.290+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1000b5cd0 con 0x7ff1000cf3a0 2023-06-23T18:04:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff0fd7fa700 1 -- 172.21.15.22:0/2755315198 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0f402a020 con 0x7ff1000cf3a0 2023-06-23T18:04:38.306 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:38.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 msgr2=0x7ff0ec0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 0x7ff0ec0687d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1000c9640 tx=0x7ff0f0009240 comp rx=0 tx=0).stop 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 msgr2=0x7ff1000b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 0x7ff1000b5fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4015000 tx=0x7ff0f4004770 comp rx=0 tx=0).stop 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 shutdown_connections 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1000d2360 0x7ff100159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000d1f30 0x7ff1000b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff0ec066320 0x7ff0ec0687d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 --2- 172.21.15.22:0/2755315198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000cf3a0 0x7ff1000b5fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 >> 172.21.15.22:0/2755315198 conn(0x7ff10006a280 msgr2=0x7ff10015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 shutdown_connections 2023-06-23T18:04:38.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.294+0000 7ff1058fe700 1 -- 172.21.15.22:0/2755315198 wait complete. 2023-06-23T18:04:38.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:38.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.430+0000 7f2662903700 1 Processor -- start 2023-06-23T18:04:38.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.430+0000 7f2662903700 1 -- start start 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f265c0cca20 0x7f265c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0d21d0 0x7f265c0d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265c0d1d40 con 0x7f265c0b1920 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0b1cf0 con 0x7f265c0cca20 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0cf130 con 0x7f265c0d21d0 2023-06-23T18:04:38.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f265c0cca20 0x7f265c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f265c0cca20 0x7f265c0d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46490/0 (socket says 172.21.15.22:46490) 2023-06-23T18:04:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 -- 172.21.15.22:0/3003662197 learned_addr learned my addr 172.21.15.22:0/3003662197 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662102700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0d21d0 0x7f265c0d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 -- 172.21.15.22:0/3003662197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0d21d0 msgr2=0x7f265c0d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0d21d0 0x7f265c0d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 -- 172.21.15.22:0/3003662197 >> v1:172.21.15.169:6789/0 conn(0x7f265c0b1920 legacy=0x7f265c0cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 -- 172.21.15.22:0/3003662197 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0cf410 con 0x7f265c0cca20 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2661100700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0cca20 0x7f265c0d06b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2650009910 tx=0x7f2650009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becb09230c609fa9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f26608ff700 1 -- 172.21.15.22:0/3003662197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011830 con 0x7f265c0cca20 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f26608ff700 1 -- 172.21.15.22:0/3003662197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650011990 con 0x7f265c0cca20 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f26608ff700 1 -- 172.21.15.22:0/3003662197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011d40 con 0x7f265c0cca20 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- 172.21.15.22:0/3003662197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0cca20 msgr2=0x7f265c0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0cca20 0x7f265c0d06b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2650009910 tx=0x7f2650009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- 172.21.15.22:0/3003662197 shutdown_connections 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0d21d0 0x7f265c0d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 --2- 172.21.15.22:0/3003662197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0cca20 0x7f265c0d06b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- 172.21.15.22:0/3003662197 >> 172.21.15.22:0/3003662197 conn(0x7f265c069960 msgr2=0x7f265c069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- 172.21.15.22:0/3003662197 shutdown_connections 2023-06-23T18:04:38.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.434+0000 7f2662903700 1 -- 172.21.15.22:0/3003662197 wait complete. 2023-06-23T18:04:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 Processor -- start 2023-06-23T18:04:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 -- start start 2023-06-23T18:04:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b1920 0x7f265c1f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 0x7f265c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0cc0e0 con 0x7f265c0d21d0 2023-06-23T18:04:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1f5c60 con 0x7f265c0cca20 2023-06-23T18:04:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662903700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b7530 con 0x7f265c0b1920 2023-06-23T18:04:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662102700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 0x7f265c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662102700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46500/0 (socket says 172.21.15.22:46500) 2023-06-23T18:04:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662102700 1 -- 172.21.15.22:0/1852012236 learned_addr learned my addr 172.21.15.22:0/1852012236 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661901700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b1920 0x7f265c1f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 -- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b1920 msgr2=0x7f265c1f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b1920 0x7f265c1f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 -- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 msgr2=0x7f265c0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2661100700 1 -- 172.21.15.22:0/1852012236 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650009570 con 0x7f265c0cca20 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.438+0000 7f2662102700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f2661100700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 0x7f265c0b5b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2650016000 tx=0x7f2650004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650004510 con 0x7f265c0cca20 2023-06-23T18:04:38.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265001f070 con 0x7f265c0cca20 2023-06-23T18:04:38.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0b7810 con 0x7f265c0cca20 2023-06-23T18:04:38.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650003900 con 0x7f265c0cca20 2023-06-23T18:04:38.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c0b5610 con 0x7f265c0cca20 2023-06-23T18:04:38.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c04e4c0 con 0x7f265c0cca20 2023-06-23T18:04:38.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.442+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2650003a60 con 0x7f265c0cca20 2023-06-23T18:04:38.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.446+0000 7f264e7fc700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 0x7f26400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.446+0000 7f2661901700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 0x7f26400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.446+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2650021070 con 0x7f265c0cca20 2023-06-23T18:04:38.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.446+0000 7f2661901700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 0x7f26400687a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2658000f60 tx=0x7f2658009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.454+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2650052620 con 0x7f265c0cca20 2023-06-23T18:04:38.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.626+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f265c0b46d0 con 0x7f265c0cca20 2023-06-23T18:04:38.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.626+0000 7f264e7fc700 1 -- 172.21.15.22:0/1852012236 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2650006da0 con 0x7f265c0cca20 2023-06-23T18:04:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 msgr2=0x7f26400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 0x7f26400687a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2658000f60 tx=0x7f2658009040 comp rx=0 tx=0).stop 2023-06-23T18:04:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 msgr2=0x7f265c0b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 0x7f265c0b5b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2650016000 tx=0x7f2650004770 comp rx=0 tx=0).stop 2023-06-23T18:04:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 shutdown_connections 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c0d21d0 0x7f265c0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f26400662f0 0x7f26400687a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cca20 0x7f265c0b5b30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 --2- 172.21.15.22:0/1852012236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b1920 0x7f265c1f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 >> 172.21.15.22:0/1852012236 conn(0x7f265c069960 msgr2=0x7f265c15a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 shutdown_connections 2023-06-23T18:04:38.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.630+0000 7f2662903700 1 -- 172.21.15.22:0/1852012236 wait complete. 2023-06-23T18:04:38.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:38.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 Processor -- start 2023-06-23T18:04:38.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 -- start start 2023-06-23T18:04:38.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6580710a0 0x7fa658071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa658156540 0x7fa6580b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa658160140 con 0x7fa6581605d0 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65815f850 con 0x7fa6580710a0 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.766+0000 7fa65ca40700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580719b0 con 0x7fa658156540 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa64ffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa6581605d0 0x7fa6580ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51662/0 (socket says 172.21.15.22:51662) 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa64ffff700 1 -- 172.21.15.22:0/2686185949 learned_addr learned my addr 172.21.15.22:0/2686185949 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa658156540 0x7fa6580b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa6577fe700 1 --2- 172.21.15.22:0/2686185949 >> v2:172.21.15.169:3300/0 conn(0x7fa6580710a0 0x7fa658071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601477357 0 0) 0x7fa658160140 con 0x7fa6581605d0 2023-06-23T18:04:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6380035d0 con 0x7fa6581605d0 2023-06-23T18:04:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182749451 0 0) 0x7fa6380035d0 con 0x7fa6581605d0 2023-06-23T18:04:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa658156540 msgr2=0x7fa6580b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa658156540 0x7fa6580b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 >> v2:172.21.15.169:3300/0 conn(0x7fa6580710a0 msgr2=0x7fa658071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 --2- 172.21.15.22:0/2686185949 >> v2:172.21.15.169:3300/0 conn(0x7fa6580710a0 0x7fa658071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580cee40 con 0x7fa6581605d0 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7fa640001670 con 0x7fa6581605d0 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 >> v1:172.21.15.22:6789/0 conn(0x7fa6581605d0 legacy=0x7fa6580ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa638003a20 0x7fa638003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004400 0x7fa6380048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638004f40 0x7fa63800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65815f850 con 0x7fa638004f40 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580719b0 con 0x7fa638004400 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63800bf70 con 0x7fa638003a20 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa658156540 0x7fa6580b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa6577fe700 1 --2- 172.21.15.22:0/2686185949 >> v2:172.21.15.169:3300/0 conn(0x7fa6580710a0 0x7fa658071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:38.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638004f40 0x7fa63800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa64ffff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004400 0x7fa6380048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa6577fe700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa638003a20 0x7fa638003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 -- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa638003a20 msgr2=0x7fa638003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa638003a20 0x7fa638003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 -- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004400 msgr2=0x7fa6380048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004400 0x7fa6380048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 -- 172.21.15.22:0/2686185949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380035d0 con 0x7fa638004f40 2023-06-23T18:04:38.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa6577fe700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa638003a20 0x7fa638003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:38.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638004f40 0x7fa63800ba30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6580aeb50 tx=0x7fa6380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4f8101bbffb259 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650003c40 con 0x7fa638004f40 2023-06-23T18:04:38.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65000f040 con 0x7fa638004f40 2023-06-23T18:04:38.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa656ffd700 1 -- 172.21.15.22:0/2686185949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000ea30 con 0x7fa638004f40 2023-06-23T18:04:38.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 -- 172.21.15.22:0/2686185949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638004f40 msgr2=0x7fa63800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 --2- 172.21.15.22:0/2686185949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638004f40 0x7fa63800ba30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6580aeb50 tx=0x7fa6380058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa657fff700 1 -- 172.21.15.22:0/2686185949 reap_dead start 2023-06-23T18:04:38.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 -- 172.21.15.22:0/2686185949 shutdown_connections 2023-06-23T18:04:38.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 -- 172.21.15.22:0/2686185949 >> 172.21.15.22:0/2686185949 conn(0x7fa65806aa70 msgr2=0x7fa65806ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 -- 172.21.15.22:0/2686185949 shutdown_connections 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.770+0000 7fa65ca40700 1 -- 172.21.15.22:0/2686185949 wait complete. 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 Processor -- start 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- start start 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004f40 0x7fa6580b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6581f4c60 0x7fa6581f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6581f5610 con 0x7fa6580aeb80 2023-06-23T18:04:38.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580afd30 con 0x7fa638004f40 2023-06-23T18:04:38.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580c40d0 con 0x7fa6581f4c60 2023-06-23T18:04:38.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46512/0 (socket says 172.21.15.22:46512) 2023-06-23T18:04:38.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 -- 172.21.15.22:0/4158138572 learned_addr learned my addr 172.21.15.22:0/4158138572 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa657fff700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6581f4c60 0x7fa6581f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa6577fe700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004f40 0x7fa6580b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 -- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6581f4c60 msgr2=0x7fa6581f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6581f4c60 0x7fa6581f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 -- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004f40 msgr2=0x7fa6580b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004f40 0x7fa6580b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 -- 172.21.15.22:0/4158138572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580af120 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa64ffff700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa640007340 tx=0x7fa640007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011580 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6400116e0 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6580af400 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011a90 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.774+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6580af9c0 con 0x7fa6580aeb80 2023-06-23T18:04:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.778+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6580afea0 con 0x7fa6580aeb80 2023-06-23T18:04:38.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.778+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa640011bf0 con 0x7fa6580aeb80 2023-06-23T18:04:38.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.778+0000 7fa654ff9700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 0x7fa63c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:38.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.778+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa640083eb0 con 0x7fa6580aeb80 2023-06-23T18:04:38.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.778+0000 7fa6577fe700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 0x7fa63c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:38.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.782+0000 7fa6577fe700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 0x7fa63c0687d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa648004670 tx=0x7fa648008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:38.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.786+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa640051120 con 0x7fa6580aeb80 2023-06-23T18:04:38.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.962+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa65804e4c0 con 0x7fa6580aeb80 2023-06-23T18:04:38.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.962+0000 7fa654ff9700 1 -- 172.21.15.22:0/4158138572 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa640054740 con 0x7fa6580aeb80 2023-06-23T18:04:38.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 msgr2=0x7fa63c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 0x7fa63c0687d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa648004670 tx=0x7fa648008040 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 msgr2=0x7fa6580c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa640007340 tx=0x7fa640007650 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 shutdown_connections 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6581f4c60 0x7fa6581f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6580aeb80 0x7fa6580c3b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa63c066320 0x7fa63c0687d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 --2- 172.21.15.22:0/4158138572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa638004f40 0x7fa6580b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:38.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 >> 172.21.15.22:0/4158138572 conn(0x7fa65806aa70 msgr2=0x7fa6580b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:38.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 shutdown_connections 2023-06-23T18:04:38.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:38.966+0000 7fa65ca40700 1 -- 172.21.15.22:0/4158138572 wait complete. 2023-06-23T18:04:38.981 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:38.990 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi169.front.sepia.ceph.com... 2023-06-23T18:04:38.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:38.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-06-23T18:04:39.023 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:39.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:04:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:39.024 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-23T18:04:39.061 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:39.062 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:39.070 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:39.070 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:39.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:39.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 Processor -- start 2023-06-23T18:04:39.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 -- start start 2023-06-23T18:04:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1e400cd5e0 0x7f1e400ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400cca70 con 0x7f1e400cd160 2023-06-23T18:04:39.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400c9780 con 0x7f1e40157f00 2023-06-23T18:04:39.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.182+0000 7f1e3f59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e400cda60 con 0x7f1e400cd5e0 2023-06-23T18:04:39.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1e400cd160 0x7f1e400c9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51670/0 (socket says 172.21.15.22:51670) 2023-06-23T18:04:39.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 -- 172.21.15.22:0/2844279072 learned_addr learned my addr 172.21.15.22:0/2844279072 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:39.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/2844279072 >> v2:172.21.15.169:3300/0 conn(0x7f1e400cd5e0 0x7f1e400ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844869909 0 0) 0x7f1e400cca70 con 0x7f1e400cd160 2023-06-23T18:04:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e2c0046e0 con 0x7f1e400cd160 2023-06-23T18:04:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378640178 0 0) 0x7f1e400c9780 con 0x7f1e40157f00 2023-06-23T18:04:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e400cca70 con 0x7f1e40157f00 2023-06-23T18:04:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727630881 0 0) 0x7f1e2c0046e0 con 0x7f1e400cd160 2023-06-23T18:04:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 >> v2:172.21.15.169:3300/0 conn(0x7f1e400cd5e0 msgr2=0x7f1e400ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 --2- 172.21.15.22:0/2844279072 >> v2:172.21.15.169:3300/0 conn(0x7f1e400cd5e0 0x7f1e400ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 >> v1:172.21.15.169:6789/0 conn(0x7f1e40157f00 legacy=0x7f1e400cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e400c4c50 con 0x7f1e400cd160 2023-06-23T18:04:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f1e30001670 con 0x7f1e400cd160 2023-06-23T18:04:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/2844279072 >> v2:172.21.15.169:3300/0 conn(0x7f1e400cd5e0 0x7f1e400ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 >> v1:172.21.15.22:6789/0 conn(0x7f1e400cd160 legacy=0x7f1e400c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e2c004d10 0x7f1e2c005100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e2c005df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e2c006380 0x7f1e2c00cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e400cda60 con 0x7f1e2c006380 2023-06-23T18:04:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c00d290 con 0x7f1e2c005980 2023-06-23T18:04:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c00d450 con 0x7f1e2c004d10 2023-06-23T18:04:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e2c006380 0x7f1e2c00cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e2c005df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3e59c700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e2c004d10 0x7f1e2c005100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 -- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e2c004d10 msgr2=0x7f1e2c005100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e2c004d10 0x7f1e2c005100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 -- 172.21.15.22:0/2844279072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e2c006380 msgr2=0x7f1e2c00cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e2c006380 0x7f1e2c00cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 -- 172.21.15.22:0/2844279072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0046e0 con 0x7f1e2c005980 2023-06-23T18:04:39.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e2c006380 0x7f1e2c00cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:39.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3ed9d700 1 -- 172.21.15.22:0/2844279072 reap_dead start 2023-06-23T18:04:39.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e37fff700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e2c005df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c0e0 tx=0x7f1e2c0069b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a38d57f29e4f11 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000cc80 con 0x7f1e2c005980 2023-06-23T18:04:39.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000cde0 con 0x7f1e2c005980 2023-06-23T18:04:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.186+0000 7f1e3dd9b700 1 -- 172.21.15.22:0/2844279072 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30005bc0 con 0x7f1e2c005980 2023-06-23T18:04:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 msgr2=0x7f1e2c005df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e2c005df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c0e0 tx=0x7f1e2c0069b0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- 172.21.15.22:0/2844279072 shutdown_connections 2023-06-23T18:04:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/2844279072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e2c005df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- 172.21.15.22:0/2844279072 >> 172.21.15.22:0/2844279072 conn(0x7f1e400cb2a0 msgr2=0x7f1e400cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- 172.21.15.22:0/2844279072 shutdown_connections 2023-06-23T18:04:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- 172.21.15.22:0/2844279072 wait complete. 2023-06-23T18:04:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 Processor -- start 2023-06-23T18:04:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- start start 2023-06-23T18:04:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e40156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 0x7f1e400712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40159640 con 0x7f1e40157480 2023-06-23T18:04:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401585e0 con 0x7f1e2c005980 2023-06-23T18:04:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40158880 con 0x7f1e4014d8f0 2023-06-23T18:04:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46530/0 (socket says 172.21.15.22:46530) 2023-06-23T18:04:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e37fff700 1 -- 172.21.15.22:0/70170325 learned_addr learned my addr 172.21.15.22:0/70170325 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3e59c700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e40156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.190+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 0x7f1e400712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 -- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 msgr2=0x7f1e400712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 0x7f1e400712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 -- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 msgr2=0x7f1e40156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e40156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 -- 172.21.15.22:0/70170325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40158b60 con 0x7f1e40157480 2023-06-23T18:04:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e3e59c700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e40156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e3ed9d700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 0x7f1e400712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e37fff700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00d420 tx=0x7f1e400c9910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30004ca0 con 0x7f1e40157480 2023-06-23T18:04:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30004e00 con 0x7f1e40157480 2023-06-23T18:04:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e40158e40 con 0x7f1e40157480 2023-06-23T18:04:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30007310 con 0x7f1e40157480 2023-06-23T18:04:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e40152b70 con 0x7f1e40157480 2023-06-23T18:04:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4004e4c0 con 0x7f1e40157480 2023-06-23T18:04:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.194+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f1e30007470 con 0x7f1e40157480 2023-06-23T18:04:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.198+0000 7f1e36ffd700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 0x7f1e200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.198+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e300848f0 con 0x7f1e40157480 2023-06-23T18:04:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.198+0000 7f1e3e59c700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 0x7f1e200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.198+0000 7f1e3e59c700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 0x7f1e200687a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007fd0 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.206+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f1e30051b70 con 0x7f1e40157480 2023-06-23T18:04:39.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.370+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e401589c0 con 0x7f1e40157480 2023-06-23T18:04:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e36ffd700 1 -- 172.21.15.22:0/70170325 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f1e401589c0 con 0x7f1e40157480 2023-06-23T18:04:39.387 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:39.387 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 msgr2=0x7f1e200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 0x7f1e200687a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007fd0 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2023-06-23T18:04:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 msgr2=0x7f1e40070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00d420 tx=0x7f1e400c9910 comp rx=0 tx=0).stop 2023-06-23T18:04:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 shutdown_connections 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e4014d8f0 0x7f1e400712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e40157480 0x7f1e40070db0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f1e200662f0 0x7f1e200687a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 --2- 172.21.15.22:0/70170325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e2c005980 0x7f1e40156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 >> 172.21.15.22:0/70170325 conn(0x7f1e400cb2a0 msgr2=0x7f1e400b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 shutdown_connections 2023-06-23T18:04:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.374+0000 7f1e3f59e700 1 -- 172.21.15.22:0/70170325 wait complete. 2023-06-23T18:04:39.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:39.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.514+0000 7f790285c700 1 Processor -- start 2023-06-23T18:04:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.514+0000 7f790285c700 1 -- start start 2023-06-23T18:04:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78f4038560 0x7f78f403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78f403b160 0x7f78f4057210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f4007360 con 0x7f78f40e29f0 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f40e2dc0 con 0x7f78f403b160 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f4053f70 con 0x7f78f4038560 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78f403b160 0x7f78f4057210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790185a700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f78f40e29f0 0x7f78f4055490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59128/0 (socket says 172.21.15.22:59128) 2023-06-23T18:04:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7901059700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78f4038560 0x7f78f403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f78f403b160 0x7f78f4057210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46534/0 (socket says 172.21.15.22:46534) 2023-06-23T18:04:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790185a700 1 -- 172.21.15.22:0/1964692365 learned_addr learned my addr 172.21.15.22:0/1964692365 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7900858700 1 -- 172.21.15.22:0/1964692365 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927534803 0 0) 0x7f78f4007360 con 0x7f78f40e29f0 2023-06-23T18:04:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7900858700 1 -- 172.21.15.22:0/1964692365 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e80035d0 con 0x7f78f40e29f0 2023-06-23T18:04:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 -- 172.21.15.22:0/1964692365 >> v2:172.21.15.169:3300/0 conn(0x7f78f4038560 msgr2=0x7f78f403ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 --2- 172.21.15.22:0/1964692365 >> v2:172.21.15.169:3300/0 conn(0x7f78f4038560 0x7f78f403ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 -- 172.21.15.22:0/1964692365 >> v1:172.21.15.169:6789/0 conn(0x7f78f40e29f0 legacy=0x7f78f4055490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 -- 172.21.15.22:0/1964692365 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f403b5a0 con 0x7f78f403b160 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790205b700 1 --2- 172.21.15.22:0/1964692365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f403b160 0x7f78f4057210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f78fc02c8d0 tx=0x7f78fc0674a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a8c1bde7ed9a69 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7900858700 1 -- 172.21.15.22:0/1964692365 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc073810 con 0x7f78f403b160 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7900858700 1 -- 172.21.15.22:0/1964692365 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc073970 con 0x7f78f403b160 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f7900858700 1 -- 172.21.15.22:0/1964692365 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc06da30 con 0x7f78f403b160 2023-06-23T18:04:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- 172.21.15.22:0/1964692365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f403b160 msgr2=0x7f78f4057210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 --2- 172.21.15.22:0/1964692365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f403b160 0x7f78f4057210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f78fc02c8d0 tx=0x7f78fc0674a0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- 172.21.15.22:0/1964692365 shutdown_connections 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 --2- 172.21.15.22:0/1964692365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f403b160 0x7f78f4057210 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 --2- 172.21.15.22:0/1964692365 >> v2:172.21.15.169:3300/0 conn(0x7f78f4038560 0x7f78f403ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- 172.21.15.22:0/1964692365 >> 172.21.15.22:0/1964692365 conn(0x7f78f4055c20 msgr2=0x7f78f4056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- 172.21.15.22:0/1964692365 shutdown_connections 2023-06-23T18:04:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.518+0000 7f790285c700 1 -- 172.21.15.22:0/1964692365 wait complete. 2023-06-23T18:04:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 Processor -- start 2023-06-23T18:04:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 -- start start 2023-06-23T18:04:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 0x7f78f4050270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f403b160 0x7f78f404ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 0x7f78f40dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4051570 con 0x7f78f40e29f0 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40507b0 con 0x7f78f4038560 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790285c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4050a10 con 0x7f78f403b160 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790205b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 0x7f78f40dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790185a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 0x7f78f4050270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790205b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 0x7f78f40dbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46540/0 (socket says 172.21.15.22:46540) 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f790205b700 1 -- 172.21.15.22:0/436220201 learned_addr learned my addr 172.21.15.22:0/436220201 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:39.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.522+0000 7f7901059700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f403b160 0x7f78f404ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 -- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f403b160 msgr2=0x7f78f404ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f403b160 0x7f78f404ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 -- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 msgr2=0x7f78f40dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 0x7f78f40dbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 -- 172.21.15.22:0/436220201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4050cf0 con 0x7f78f4038560 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790185a700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 0x7f78f4050270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c4d0 tx=0x7f78f800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005d90 con 0x7f78f4038560 2023-06-23T18:04:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f800ce80 con 0x7f78f4038560 2023-06-23T18:04:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f4050fd0 con 0x7f78f4038560 2023-06-23T18:04:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80072f0 con 0x7f78f4038560 2023-06-23T18:04:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f404f490 con 0x7f78f4038560 2023-06-23T18:04:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f404f680 con 0x7f78f4038560 2023-06-23T18:04:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.526+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f78f8007450 con 0x7f78f4038560 2023-06-23T18:04:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.530+0000 7f78f27fc700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.530+0000 7f7901059700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.530+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f78f80833e0 con 0x7f78f4038560 2023-06-23T18:04:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.530+0000 7f7901059700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 0x7f78e00687a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78f40536c0 tx=0x7f78ec009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.538+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f78f8050490 con 0x7f78f4038560 2023-06-23T18:04:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78f4051a00 con 0x7f78f4038560 2023-06-23T18:04:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f78f27fc700 1 -- 172.21.15.22:0/436220201 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78f4051a00 con 0x7f78f4038560 2023-06-23T18:04:39.715 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 msgr2=0x7f78e00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 0x7f78e00687a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78f40536c0 tx=0x7f78ec009240 comp rx=0 tx=0).stop 2023-06-23T18:04:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 msgr2=0x7f78f4050270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 0x7f78f4050270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c4d0 tx=0x7f78f800c7e0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.702+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 shutdown_connections 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78f40e29f0 0x7f78f40dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78f403b160 0x7f78f404ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f78e00662f0 0x7f78e00687a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 --2- 172.21.15.22:0/436220201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78f4038560 0x7f78f4050270 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 >> 172.21.15.22:0/436220201 conn(0x7f78f4055c20 msgr2=0x7f78f403bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:39.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 shutdown_connections 2023-06-23T18:04:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.706+0000 7f790285c700 1 -- 172.21.15.22:0/436220201 wait complete. 2023-06-23T18:04:39.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.842+0000 7fd6751cf700 1 Processor -- start 2023-06-23T18:04:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.842+0000 7fd6751cf700 1 -- start start 2023-06-23T18:04:39.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.842+0000 7fd6751cf700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd6700d1f30 0x7fd6700d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.842+0000 7fd6751cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d2360 0x7fd6700d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.842+0000 7fd6751cf700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6700ccb30 con 0x7fd6700cf3a0 2023-06-23T18:04:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700d2880 con 0x7fd6700d1f30 2023-06-23T18:04:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700d2b60 con 0x7fd6700d2360 2023-06-23T18:04:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd6700d1f30 0x7fd6700d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66ffff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd6700cf3a0 0x7fd6700cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59142/0 (socket says 172.21.15.22:59142) 2023-06-23T18:04:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66ffff700 1 -- 172.21.15.22:0/1882265294 learned_addr learned my addr 172.21.15.22:0/1882265294 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6749ce700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d2360 0x7fd6700d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 -- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d2360 msgr2=0x7fd6700d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d2360 0x7fd6700d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 -- 172.21.15.22:0/1882265294 >> v1:172.21.15.169:6789/0 conn(0x7fd6700cf3a0 legacy=0x7fd6700cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 -- 172.21.15.22:0/1882265294 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700d2e40 con 0x7fd6700d1f30 2023-06-23T18:04:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66f7fe700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d1f30 0x7fd6700d0020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd658009850 tx=0x7fd658009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb3a43622f8df91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66effd700 1 -- 172.21.15.22:0/1882265294 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011830 con 0x7fd6700d1f30 2023-06-23T18:04:39.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66effd700 1 -- 172.21.15.22:0/1882265294 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658011990 con 0x7fd6700d1f30 2023-06-23T18:04:39.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd66effd700 1 -- 172.21.15.22:0/1882265294 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011d40 con 0x7fd6700d1f30 2023-06-23T18:04:39.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d1f30 msgr2=0x7fd6700d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d1f30 0x7fd6700d0020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd658009850 tx=0x7fd658009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- 172.21.15.22:0/1882265294 shutdown_connections 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d2360 0x7fd6700d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 --2- 172.21.15.22:0/1882265294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d1f30 0x7fd6700d0020 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- 172.21.15.22:0/1882265294 >> 172.21.15.22:0/1882265294 conn(0x7fd67006a280 msgr2=0x7fd67009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- 172.21.15.22:0/1882265294 shutdown_connections 2023-06-23T18:04:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.846+0000 7fd6751cf700 1 -- 172.21.15.22:0/1882265294 wait complete. 2023-06-23T18:04:39.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 Processor -- start 2023-06-23T18:04:39.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 -- start start 2023-06-23T18:04:39.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d1f30 0x7fd6700b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d2360 0x7fd6700b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670154040 con 0x7fd6700cf3a0 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701f5070 con 0x7fd6700d2360 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6751cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700b5960 con 0x7fd6700d1f30 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46542/0 (socket says 172.21.15.22:46542) 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 -- 172.21.15.22:0/4081818570 learned_addr learned my addr 172.21.15.22:0/4081818570 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:39.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66f7fe700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d1f30 0x7fd6700b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd6749ce700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d2360 0x7fd6700b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 -- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d1f30 msgr2=0x7fd6700b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d1f30 0x7fd6700b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 -- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d2360 msgr2=0x7fd6700b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d2360 0x7fd6700b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.850+0000 7fd66ffff700 1 -- 172.21.15.22:0/4081818570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580094b0 con 0x7fd6700cf3a0 2023-06-23T18:04:39.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66ffff700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd66000cb40 tx=0x7fd66000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000a9e0 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000ab40 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6700b5c40 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660012400 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6700b7690 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6701f5290 con 0x7fd6700cf3a0 2023-06-23T18:04:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd660005d50 con 0x7fd6700cf3a0 2023-06-23T18:04:39.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 0x7fd6500687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:39.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd660083820 con 0x7fd6700cf3a0 2023-06-23T18:04:39.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.854+0000 7fd66f7fe700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 0x7fd6500687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:39.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.858+0000 7fd66f7fe700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 0x7fd6500687a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd658008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:39.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:39.866+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd6701f5290 con 0x7fd6700cf3a0 2023-06-23T18:04:40.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.026+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6700b6160 con 0x7fd6700cf3a0 2023-06-23T18:04:40.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.026+0000 7fd66cff9700 1 -- 172.21.15.22:0/4081818570 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd660050a20 con 0x7fd6700cf3a0 2023-06-23T18:04:40.040 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 msgr2=0x7fd6500687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 0x7fd6500687a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd658008030 comp rx=0 tx=0).stop 2023-06-23T18:04:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 msgr2=0x7fd6700b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd66000cb40 tx=0x7fd66000ce50 comp rx=0 tx=0).stop 2023-06-23T18:04:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 shutdown_connections 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd6500662f0 0x7fd6500687a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6700d2360 0x7fd6700b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6700d1f30 0x7fd6700b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 --2- 172.21.15.22:0/4081818570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6700cf3a0 0x7fd6700b4650 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 >> 172.21.15.22:0/4081818570 conn(0x7fd67006a280 msgr2=0x7fd67006c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 shutdown_connections 2023-06-23T18:04:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.030+0000 7fd6751cf700 1 -- 172.21.15.22:0/4081818570 wait complete. 2023-06-23T18:04:40.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.166+0000 7ff26392e700 1 Processor -- start 2023-06-23T18:04:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.166+0000 7ff26392e700 1 -- start start 2023-06-23T18:04:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 0x7ff25c0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff25c0b53c0 0x7ff25c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0b5d00 con 0x7ff25c0b4730 2023-06-23T18:04:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0d0420 con 0x7ff25c0b53c0 2023-06-23T18:04:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0cd670 con 0x7ff25c15f3b0 2023-06-23T18:04:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff25c0b53c0 0x7ff25c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff25c0b53c0 0x7ff25c0b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46552/0 (socket says 172.21.15.22:46552) 2023-06-23T18:04:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 -- 172.21.15.22:0/2359324438 learned_addr learned my addr 172.21.15.22:0/2359324438 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff260ec9700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff25c0b4730 0x7ff25c0ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51682/0 (socket says 172.21.15.22:51682) 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff2616ca700 1 --2- 172.21.15.22:0/2359324438 >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 0x7ff25c0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 -- 172.21.15.22:0/2359324438 >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 msgr2=0x7ff25c0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 --2- 172.21.15.22:0/2359324438 >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 0x7ff25c0d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 -- 172.21.15.22:0/2359324438 >> v1:172.21.15.22:6789/0 conn(0x7ff25c0b4730 legacy=0x7ff25c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 -- 172.21.15.22:0/2359324438 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0cd950 con 0x7ff25c0b53c0 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff2616ca700 1 --2- 172.21.15.22:0/2359324438 >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 0x7ff25c0d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff261ecb700 1 --2- 172.21.15.22:0/2359324438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c0b57c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff25c08b890 tx=0x7ff25400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b1b3450a1ac9e2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff253fff700 1 -- 172.21.15.22:0/2359324438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25400e070 con 0x7ff25c0b53c0 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff253fff700 1 -- 172.21.15.22:0/2359324438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25400b9c0 con 0x7ff25c0b53c0 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff253fff700 1 -- 172.21.15.22:0/2359324438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25400bd70 con 0x7ff25c0b53c0 2023-06-23T18:04:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- 172.21.15.22:0/2359324438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 msgr2=0x7ff25c0b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 --2- 172.21.15.22:0/2359324438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c0b57c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff25c08b890 tx=0x7ff25400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- 172.21.15.22:0/2359324438 shutdown_connections 2023-06-23T18:04:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 --2- 172.21.15.22:0/2359324438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c0b57c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 --2- 172.21.15.22:0/2359324438 >> v2:172.21.15.169:3300/0 conn(0x7ff25c15f3b0 0x7ff25c0d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- 172.21.15.22:0/2359324438 >> 172.21.15.22:0/2359324438 conn(0x7ff25c069960 msgr2=0x7ff25c069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- 172.21.15.22:0/2359324438 shutdown_connections 2023-06-23T18:04:40.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.170+0000 7ff26392e700 1 -- 172.21.15.22:0/2359324438 wait complete. 2023-06-23T18:04:40.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 Processor -- start 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 -- start start 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c0b4730 0x7ff25c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 0x7ff25c06b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c14ab20 con 0x7ff25c0b53c0 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0cc4c0 con 0x7ff25c0b4730 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff26392e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c149ea0 con 0x7ff25c15f3b0 2023-06-23T18:04:40.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46558/0 (socket says 172.21.15.22:46558) 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 -- 172.21.15.22:0/4017490356 learned_addr learned my addr 172.21.15.22:0/4017490356 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff2616ca700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c0b4730 0x7ff25c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff261ecb700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 0x7ff25c06b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 -- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 msgr2=0x7ff25c06b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 0x7ff25c06b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 -- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c0b4730 msgr2=0x7ff25c0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c0b4730 0x7ff25c0cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 -- 172.21.15.22:0/4017490356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff254009b30 con 0x7ff25c0b53c0 2023-06-23T18:04:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff261ecb700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 0x7ff25c06b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.174+0000 7ff260ec9700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c700 tx=0x7ff24c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c005cd0 con 0x7ff25c0b53c0 2023-06-23T18:04:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c005e30 con 0x7ff25c0b53c0 2023-06-23T18:04:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c14a180 con 0x7ff25c0b53c0 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0072d0 con 0x7ff25c0b53c0 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c14a740 con 0x7ff25c0b53c0 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7ff24c007430 con 0x7ff25c0b53c0 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c04dc70 con 0x7ff25c0b53c0 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 0x7ff2400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff24c083270 con 0x7ff25c0b53c0 2023-06-23T18:04:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff2616ca700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 0x7ff2400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.178+0000 7ff2616ca700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 0x7ff2400687a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c920 tx=0x7ff258008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.186+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7ff24c050480 con 0x7ff25c0b53c0 2023-06-23T18:04:40.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.354+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff25c0b6e80 con 0x7ff25c0b53c0 2023-06-23T18:04:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.354+0000 7ff251ffb700 1 -- 172.21.15.22:0/4017490356 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff25c0b6e80 con 0x7ff25c0b53c0 2023-06-23T18:04:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:40.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 msgr2=0x7ff2400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 0x7ff2400687a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c920 tx=0x7ff258008040 comp rx=0 tx=0).stop 2023-06-23T18:04:40.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 msgr2=0x7ff25c1493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c700 tx=0x7ff24c00cac0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 shutdown_connections 2023-06-23T18:04:40.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c15f3b0 0x7ff25c06b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff25c0b53c0 0x7ff25c1493c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7ff2400662f0 0x7ff2400687a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 --2- 172.21.15.22:0/4017490356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c0b4730 0x7ff25c0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 >> 172.21.15.22:0/4017490356 conn(0x7ff25c069960 msgr2=0x7ff25c15df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 shutdown_connections 2023-06-23T18:04:40.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.358+0000 7ff26392e700 1 -- 172.21.15.22:0/4017490356 wait complete. 2023-06-23T18:04:40.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 Processor -- start 2023-06-23T18:04:40.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 -- start start 2023-06-23T18:04:40.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7644155c20 0x7f76440b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764415bc30 con 0x7f76440cc3d0 2023-06-23T18:04:40.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7644070c40 con 0x7f76440cc850 2023-06-23T18:04:40.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7648a34700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7644071500 con 0x7f7644155c20 2023-06-23T18:04:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f763bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f76440cc3d0 0x7f764406fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51688/0 (socket says 172.21.15.22:51688) 2023-06-23T18:04:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f7642d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7644155c20 0x7f76440b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f763bfff700 1 -- 172.21.15.22:0/3576153307 learned_addr learned my addr 172.21.15.22:0/3576153307 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f764259c700 1 -- 172.21.15.22:0/3576153307 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358408822 0 0) 0x7f764415bc30 con 0x7f76440cc3d0 2023-06-23T18:04:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.494+0000 7f764259c700 1 -- 172.21.15.22:0/3576153307 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76300035d0 con 0x7f76440cc3d0 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7642d9d700 1 -- 172.21.15.22:0/3576153307 >> v1:172.21.15.169:6789/0 conn(0x7f76440cc850 legacy=0x7f7644070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7642d9d700 1 -- 172.21.15.22:0/3576153307 >> v1:172.21.15.22:6789/0 conn(0x7f76440cc3d0 legacy=0x7f764406fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7642d9d700 1 -- 172.21.15.22:0/3576153307 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764415be80 con 0x7f7644155c20 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7642d9d700 1 --2- 172.21.15.22:0/3576153307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76440b0da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7634009850 tx=0x7f7634009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c991838ff447b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f764259c700 1 -- 172.21.15.22:0/3576153307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011830 con 0x7f7644155c20 2023-06-23T18:04:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f764259c700 1 -- 172.21.15.22:0/3576153307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634011990 con 0x7f7644155c20 2023-06-23T18:04:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f764259c700 1 -- 172.21.15.22:0/3576153307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011d40 con 0x7f7644155c20 2023-06-23T18:04:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- 172.21.15.22:0/3576153307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 msgr2=0x7f76440b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 --2- 172.21.15.22:0/3576153307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76440b0da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7634009850 tx=0x7f7634009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- 172.21.15.22:0/3576153307 shutdown_connections 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 --2- 172.21.15.22:0/3576153307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76440b0da0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- 172.21.15.22:0/3576153307 >> 172.21.15.22:0/3576153307 conn(0x7f764406ca60 msgr2=0x7f76440ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- 172.21.15.22:0/3576153307 shutdown_connections 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- 172.21.15.22:0/3576153307 wait complete. 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 Processor -- start 2023-06-23T18:04:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.498+0000 7f7648a34700 1 -- start start 2023-06-23T18:04:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76440cc850 0x7f76440c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 0x7f764414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644149c30 con 0x7f7644155c20 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76441573e0 con 0x7f76440cc850 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c7520 con 0x7f76440c9ee0 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46582/0 (socket says 172.21.15.22:46582) 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 -- 172.21.15.22:0/3671608631 learned_addr learned my addr 172.21.15.22:0/3671608631 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7642d9d700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76440cc850 0x7f76440c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f764359e700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 0x7f764414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 -- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 msgr2=0x7f764414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 0x7f764414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 -- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76440cc850 msgr2=0x7f76440c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76440cc850 0x7f76440c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 -- 172.21.15.22:0/3671608631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340094b0 con 0x7f7644155c20 2023-06-23T18:04:40.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f764359e700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 0x7f764414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:40.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763bfff700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c4d0 tx=0x7f762c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c005d90 con 0x7f7644155c20 2023-06-23T18:04:40.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c0070d0 con 0x7f7644155c20 2023-06-23T18:04:40.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76440c7800 con 0x7f7644155c20 2023-06-23T18:04:40.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.502+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c007480 con 0x7f7644155c20 2023-06-23T18:04:40.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76440c7ae0 con 0x7f7644155c20 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76441578c0 con 0x7f7644155c20 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f762c0075e0 con 0x7f7644155c20 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f763b7fe700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 0x7f76240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f762c083440 con 0x7f7644155c20 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.506+0000 7f7642d9d700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 0x7f76240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.510+0000 7f7642d9d700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 0x7f76240687a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.514+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f76441578c0 con 0x7f7644155c20 2023-06-23T18:04:40.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.682+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76441578c0 con 0x7f7644155c20 2023-06-23T18:04:40.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.682+0000 7f763b7fe700 1 -- 172.21.15.22:0/3671608631 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76441578c0 con 0x7f7644155c20 2023-06-23T18:04:40.698 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:40.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 msgr2=0x7f76240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 0x7f76240687a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634008030 comp rx=0 tx=0).stop 2023-06-23T18:04:40.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 msgr2=0x7f76441496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c4d0 tx=0x7f762c00c7e0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 shutdown_connections 2023-06-23T18:04:40.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76440c9ee0 0x7f764414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7644155c20 0x7f76441496f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f76240662f0 0x7f76240687a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 --2- 172.21.15.22:0/3671608631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76440cc850 0x7f76440c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 >> 172.21.15.22:0/3671608631 conn(0x7f76440cc3d0 msgr2=0x7f76440c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 shutdown_connections 2023-06-23T18:04:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.686+0000 7f7648a34700 1 -- 172.21.15.22:0/3671608631 wait complete. 2023-06-23T18:04:40.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.818+0000 7fcd4e75b700 1 Processor -- start 2023-06-23T18:04:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.818+0000 7fcd4e75b700 1 -- start start 2023-06-23T18:04:40.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 0x7fcd480cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcd480d0730 0x7fcd480ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd480ccb30 con 0x7fcd480d1f30 2023-06-23T18:04:40.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd480cfcb0 con 0x7fcd480d0730 2023-06-23T18:04:40.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480d2880 con 0x7fcd480cf3a0 2023-06-23T18:04:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcd480d0730 0x7fcd480ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcd480d0730 0x7fcd480ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46596/0 (socket says 172.21.15.22:46596) 2023-06-23T18:04:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 -- 172.21.15.22:0/217846098 learned_addr learned my addr 172.21.15.22:0/217846098 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4d759700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 0x7fcd480cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 -- 172.21.15.22:0/217846098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 msgr2=0x7fcd480cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 0x7fcd480cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 -- 172.21.15.22:0/217846098 >> v1:172.21.15.169:6789/0 conn(0x7fcd480d1f30 legacy=0x7fcd480d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 -- 172.21.15.22:0/217846098 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480d2b60 con 0x7fcd480d0730 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4d759700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 0x7fcd480cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4df5a700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480ccfc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009e50 tx=0x7fcd4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34629feb9c96c63 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd3bfff700 1 -- 172.21.15.22:0/217846098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000e070 con 0x7fcd480d0730 2023-06-23T18:04:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd3bfff700 1 -- 172.21.15.22:0/217846098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4000b9c0 con 0x7fcd480d0730 2023-06-23T18:04:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd3bfff700 1 -- 172.21.15.22:0/217846098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000bd70 con 0x7fcd480d0730 2023-06-23T18:04:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- 172.21.15.22:0/217846098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 msgr2=0x7fcd480ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480ccfc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009e50 tx=0x7fcd4000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:40.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- 172.21.15.22:0/217846098 shutdown_connections 2023-06-23T18:04:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480ccfc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/217846098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480cf3a0 0x7fcd480cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- 172.21.15.22:0/217846098 >> 172.21.15.22:0/217846098 conn(0x7fcd4806a280 msgr2=0x7fcd4809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- 172.21.15.22:0/217846098 shutdown_connections 2023-06-23T18:04:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.822+0000 7fcd4e75b700 1 -- 172.21.15.22:0/217846098 wait complete. 2023-06-23T18:04:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 Processor -- start 2023-06-23T18:04:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 -- start start 2023-06-23T18:04:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd480cf3a0 0x7fcd480cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 0x7fcd480b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480c97b0 con 0x7fcd480d0730 2023-06-23T18:04:40.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480b6530 con 0x7fcd480cf3a0 2023-06-23T18:04:40.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4e75b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480b6810 con 0x7fcd480d1f30 2023-06-23T18:04:40.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4cf58700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4d759700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd480cf3a0 0x7fcd480cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4cf58700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46610/0 (socket says 172.21.15.22:46610) 2023-06-23T18:04:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4cf58700 1 -- 172.21.15.22:0/3247355124 learned_addr learned my addr 172.21.15.22:0/3247355124 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.826+0000 7fcd4df5a700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 0x7fcd480b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 -- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 msgr2=0x7fcd480b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 0x7fcd480b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 -- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd480cf3a0 msgr2=0x7fcd480cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd480cf3a0 0x7fcd480cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 -- 172.21.15.22:0/3247355124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40009b30 con 0x7fcd480d0730 2023-06-23T18:04:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4df5a700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 0x7fcd480b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4cf58700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c500 tx=0x7fcd3c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0052d0 con 0x7fcd480d0730 2023-06-23T18:04:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c005430 con 0x7fcd480d0730 2023-06-23T18:04:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd480b6af0 con 0x7fcd480d0730 2023-06-23T18:04:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0055a0 con 0x7fcd480d0730 2023-06-23T18:04:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd480b70b0 con 0x7fcd480d0730 2023-06-23T18:04:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.830+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd480b72a0 con 0x7fcd480d0730 2023-06-23T18:04:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.834+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcd3c0038e0 con 0x7fcd480d0730 2023-06-23T18:04:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.834+0000 7fcd39ffb700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 0x7fcd2c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.834+0000 7fcd4d759700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 0x7fcd2c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.834+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcd3c083470 con 0x7fcd480d0730 2023-06-23T18:04:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.834+0000 7fcd4d759700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 0x7fcd2c0687a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd440099e0 tx=0x7fcd44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.838+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fcd3c050520 con 0x7fcd480d0730 2023-06-23T18:04:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.998+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcd4804e4c0 con 0x7fcd480d0730 2023-06-23T18:04:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:40.998+0000 7fcd39ffb700 1 -- 172.21.15.22:0/3247355124 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd3c053b40 con 0x7fcd480d0730 2023-06-23T18:04:41.012 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 msgr2=0x7fcd2c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 0x7fcd2c0687a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd440099e0 tx=0x7fcd44008040 comp rx=0 tx=0).stop 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 msgr2=0x7fcd480b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c500 tx=0x7fcd3c00c810 comp rx=0 tx=0).stop 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 shutdown_connections 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd480d1f30 0x7fcd480b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd480d0730 0x7fcd480b8800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcd2c0662f0 0x7fcd2c0687a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 --2- 172.21.15.22:0/3247355124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd480cf3a0 0x7fcd480cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 >> 172.21.15.22:0/3247355124 conn(0x7fcd4806a280 msgr2=0x7fcd4806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 shutdown_connections 2023-06-23T18:04:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.002+0000 7fcd4e75b700 1 -- 172.21.15.22:0/3247355124 wait complete. 2023-06-23T18:04:41.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 Processor -- start 2023-06-23T18:04:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- start start 2023-06-23T18:04:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b100cf630 0x7f6b100cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b1015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100cd7e0 con 0x7f6b100d0e40 2023-06-23T18:04:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100d0a20 con 0x7f6b100cf630 2023-06-23T18:04:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cff40 con 0x7f6b100b2560 2023-06-23T18:04:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b100cf630 0x7f6b100cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b100cf630 0x7f6b100cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46614/0 (socket says 172.21.15.22:46614) 2023-06-23T18:04:41.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 -- 172.21.15.22:0/1700685627 learned_addr learned my addr 172.21.15.22:0/1700685627 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0f7fe700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b1015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 -- 172.21.15.22:0/1700685627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 msgr2=0x7f6b1015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b1015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 -- 172.21.15.22:0/1700685627 >> v1:172.21.15.169:6789/0 conn(0x7f6b100d0e40 legacy=0x7f6b100cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 -- 172.21.15.22:0/1700685627 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100cd500 con 0x7f6b100cf630 2023-06-23T18:04:41.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0effd700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100cf630 0x7f6b100cfa00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009910 tx=0x7f6b00009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd9cb15e7e3c857 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0dffb700 1 -- 172.21.15.22:0/1700685627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004230 con 0x7f6b100cf630 2023-06-23T18:04:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0dffb700 1 -- 172.21.15.22:0/1700685627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b00004390 con 0x7f6b100cf630 2023-06-23T18:04:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0dffb700 1 -- 172.21.15.22:0/1700685627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00009e60 con 0x7f6b100cf630 2023-06-23T18:04:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1700685627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100cf630 msgr2=0x7f6b100cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100cf630 0x7f6b100cfa00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009910 tx=0x7f6b00009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1700685627 shutdown_connections 2023-06-23T18:04:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b1015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1700685627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100cf630 0x7f6b100cfa00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.134+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1700685627 >> 172.21.15.22:0/1700685627 conn(0x7f6b10069960 msgr2=0x7f6b10069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1700685627 shutdown_connections 2023-06-23T18:04:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1700685627 wait complete. 2023-06-23T18:04:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 Processor -- start 2023-06-23T18:04:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- start start 2023-06-23T18:04:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b100b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b100cf630 0x7f6b100c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101f5040 con 0x7f6b100d0e40 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101f5320 con 0x7f6b100cf630 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101f5600 con 0x7f6b100b2560 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46620/0 (socket says 172.21.15.22:46620) 2023-06-23T18:04:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 -- 172.21.15.22:0/1734559629 learned_addr learned my addr 172.21.15.22:0/1734559629 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0effd700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b100b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0e7fc700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b100cf630 0x7f6b100c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 -- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 msgr2=0x7f6b100b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b100b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 -- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b100cf630 msgr2=0x7f6b100c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b100cf630 0x7f6b100c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 -- 172.21.15.22:0/1734559629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009570 con 0x7f6b100d0e40 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.138+0000 7f6b0f7fe700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007e60 tx=0x7f6b0800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800ede0 con 0x7f6b100d0e40 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08014420 con 0x7f6b100d0e40 2023-06-23T18:04:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b101f58e0 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080147d0 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b101f1930 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1004e4c0 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6b08014930 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 0x7f6afc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6b08084620 con 0x7f6b100d0e40 2023-06-23T18:04:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6b0effd700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 0x7f6afc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.142+0000 7f6b0effd700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 0x7f6afc0687d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009910 tx=0x7f6b00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.146+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6b08051800 con 0x7f6b100d0e40 2023-06-23T18:04:41.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.322+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b100b1320 con 0x7f6b100d0e40 2023-06-23T18:04:41.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.322+0000 7f6af77fe700 1 -- 172.21.15.22:0/1734559629 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b08054e20 con 0x7f6b100d0e40 2023-06-23T18:04:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 msgr2=0x7f6afc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 0x7f6afc0687d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009910 tx=0x7f6b00008030 comp rx=0 tx=0).stop 2023-06-23T18:04:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 msgr2=0x7f6b100c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007e60 tx=0x7f6b0800e3c0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 shutdown_connections 2023-06-23T18:04:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b100d0e40 0x7f6b100c6320 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6afc066320 0x7f6afc0687d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b100cf630 0x7f6b100c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 --2- 172.21.15.22:0/1734559629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b100b2560 0x7f6b100b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 >> 172.21.15.22:0/1734559629 conn(0x7f6b10069960 msgr2=0x7f6b1015c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 shutdown_connections 2023-06-23T18:04:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.326+0000 7f6b0ffff700 1 -- 172.21.15.22:0/1734559629 wait complete. 2023-06-23T18:04:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:41.354 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:04:41.355 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:41.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.514+0000 7f6f316da700 1 Processor -- start 2023-06-23T18:04:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.514+0000 7f6f316da700 1 -- start start 2023-06-23T18:04:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 0x7f6f2c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c0c9b50 con 0x7f6f2c070280 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c1590a0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0c9e30 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f2b7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 0x7f6f2c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:55890/0 (socket says 172.21.15.22:55890) 2023-06-23T18:04:41.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 -- 172.21.15.22:0/3348697922 learned_addr learned my addr 172.21.15.22:0/3348697922 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 -- 172.21.15.22:0/3348697922 >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 msgr2=0x7f6f2c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:41.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 --2- 172.21.15.22:0/3348697922 >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 0x7f6f2c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 -- 172.21.15.22:0/3348697922 >> v1:172.21.15.169:6789/0 conn(0x7f6f2c070280 legacy=0x7f6f2c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 -- 172.21.15.22:0/3348697922 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0ca110 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f2b7fe700 1 --2- 172.21.15.22:0/3348697922 >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 0x7f6f2c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f30ed9700 1 --2- 172.21.15.22:0/3348697922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009e50 tx=0x7f6f2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da0004530656dc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f2affd700 1 -- 172.21.15.22:0/3348697922 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400e070 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f2affd700 1 -- 172.21.15.22:0/3348697922 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400b910 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f2affd700 1 -- 172.21.15.22:0/3348697922 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bcc0 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- 172.21.15.22:0/3348697922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 msgr2=0x7f6f2c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 --2- 172.21.15.22:0/3348697922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009e50 tx=0x7f6f2400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- 172.21.15.22:0/3348697922 shutdown_connections 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 --2- 172.21.15.22:0/3348697922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c0c93a0 0x7f6f2c06b600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 --2- 172.21.15.22:0/3348697922 >> v2:172.21.15.169:3300/0 conn(0x7f6f2c0b01a0 0x7f6f2c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.518+0000 7f6f316da700 1 -- 172.21.15.22:0/3348697922 >> 172.21.15.22:0/3348697922 conn(0x7f6f2c06a5f0 msgr2=0x7f6f2c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.522+0000 7f6f316da700 1 -- 172.21.15.22:0/3348697922 shutdown_connections 2023-06-23T18:04:41.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.522+0000 7f6f316da700 1 -- 172.21.15.22:0/3348697922 wait complete. 2023-06-23T18:04:41.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.522+0000 7f6f316da700 1 Processor -- start 2023-06-23T18:04:41.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.522+0000 7f6f316da700 1 -- start start 2023-06-23T18:04:41.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c070280 0x7f6f2c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2c0c93a0 0x7f6f2c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0af370 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0b0820 con 0x7f6f2c0c93a0 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0ae030 con 0x7f6f2c070280 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46624/0 (socket says 172.21.15.22:46624) 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 -- 172.21.15.22:0/42589499 learned_addr learned my addr 172.21.15.22:0/42589499 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2bfff700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c070280 0x7f6f2c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f30ed9700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2c0c93a0 0x7f6f2c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 -- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c070280 msgr2=0x7f6f2c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c070280 0x7f6f2c0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 -- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2c0c93a0 msgr2=0x7f6f2c14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2c0c93a0 0x7f6f2c14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 -- 172.21.15.22:0/42589499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140094b0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f2b7fe700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400ebe0 tx=0x7f6f1400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400c850 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400c9b0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400cd40 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f24009b30 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.526+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c0ae5f0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c04e4c0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f6f14013470 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f28ff9700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 0x7f6f0c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6f14084580 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f2bfff700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 0x7f6f0c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.530+0000 7f6f2bfff700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 0x7f6f0c0687a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c000f60 tx=0x7f6f1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.538+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6f140515b0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.706+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f2c09a250 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.706+0000 7f6f28ff9700 1 -- 172.21.15.22:0/42589499 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f6f1400cea0 con 0x7f6f2c0b01a0 2023-06-23T18:04:41.721 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:41.722 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 msgr2=0x7f6f0c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 0x7f6f0c0687a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c000f60 tx=0x7f6f1c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 msgr2=0x7f6f2c1439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400ebe0 tx=0x7f6f1400eef0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 shutdown_connections 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f6f0c0662f0 0x7f6f0c0687a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2c0c93a0 0x7f6f2c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f2c0b01a0 0x7f6f2c1439c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 --2- 172.21.15.22:0/42589499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2c070280 0x7f6f2c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 >> 172.21.15.22:0/42589499 conn(0x7f6f2c06a5f0 msgr2=0x7f6f2c1523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 shutdown_connections 2023-06-23T18:04:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.710+0000 7f6f316da700 1 -- 172.21.15.22:0/42589499 wait complete. 2023-06-23T18:04:41.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:41.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:41.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.850+0000 7f88bea7d700 1 Processor -- start 2023-06-23T18:04:41.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.850+0000 7f88bea7d700 1 -- start start 2023-06-23T18:04:41.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88b806df70 0x7f88b80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b815a0a0 con 0x7f88b8159cd0 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b806e3f0 con 0x7f88b806df70 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80b1740 con 0x7f88b80b1f70 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bd27a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88b806df70 0x7f88b80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bd27a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b806da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:55910/0 (socket says 172.21.15.22:55910) 2023-06-23T18:04:41.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bd27a700 1 -- 172.21.15.22:0/2984880091 learned_addr learned my addr 172.21.15.22:0/2984880091 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bda7b700 1 --1- 172.21.15.22:0/2984880091 >> v1:172.21.15.169:6789/0 conn(0x7f88b8159cd0 0x7f88b806e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59194/0 (socket says 172.21.15.22:59194) 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 -- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 msgr2=0x7f88b806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 --2- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b806da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 -- 172.21.15.22:0/2984880091 >> v1:172.21.15.169:6789/0 conn(0x7f88b8159cd0 legacy=0x7f88b806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 -- 172.21.15.22:0/2984880091 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80cae40 con 0x7f88b806df70 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88be27c700 1 --2- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b806df70 0x7f88b80cbc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d90 tx=0x7f88b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1fcfc69e854200 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bca79700 1 -- 172.21.15.22:0/2984880091 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b000e070 con 0x7f88b806df70 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bca79700 1 -- 172.21.15.22:0/2984880091 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b000bb20 con 0x7f88b806df70 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bca79700 1 -- 172.21.15.22:0/2984880091 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0012430 con 0x7f88b806df70 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b806df70 msgr2=0x7f88b80cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 --2- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b806df70 0x7f88b80cbc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d90 tx=0x7f88b000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- 172.21.15.22:0/2984880091 shutdown_connections 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 --2- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b806df70 0x7f88b80cbc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 --2- 172.21.15.22:0/2984880091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b806da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- 172.21.15.22:0/2984880091 >> 172.21.15.22:0/2984880091 conn(0x7f88b806b700 msgr2=0x7f88b80b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:41.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- 172.21.15.22:0/2984880091 shutdown_connections 2023-06-23T18:04:41.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.854+0000 7f88bea7d700 1 -- 172.21.15.22:0/2984880091 wait complete. 2023-06-23T18:04:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 Processor -- start 2023-06-23T18:04:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 -- start start 2023-06-23T18:04:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b81efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8159cd0 0x7f88b814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b814ff10 con 0x7f88b806df70 2023-06-23T18:04:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b815d4b0 con 0x7f88b8159cd0 2023-06-23T18:04:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bea7d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b814f300 con 0x7f88b80b1f70 2023-06-23T18:04:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46638/0 (socket says 172.21.15.22:46638) 2023-06-23T18:04:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 -- 172.21.15.22:0/3306083861 learned_addr learned my addr 172.21.15.22:0/3306083861 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bd27a700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b81efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88be27c700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8159cd0 0x7f88b814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 -- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 msgr2=0x7f88b81efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b81efd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 -- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8159cd0 msgr2=0x7f88b814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8159cd0 0x7f88b814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.858+0000 7f88bda7b700 1 -- 172.21.15.22:0/3306083861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0009a70 con 0x7f88b806df70 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88bda7b700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c410 tx=0x7f88b400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4005cd0 con 0x7f88b806df70 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4005e30 con 0x7f88b806df70 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b814f5e0 con 0x7f88b806df70 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40072f0 con 0x7f88b806df70 2023-06-23T18:04:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b814fba0 con 0x7f88b806df70 2023-06-23T18:04:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b804e4c0 con 0x7f88b806df70 2023-06-23T18:04:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f88b4007450 con 0x7f88b806df70 2023-06-23T18:04:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.862+0000 7f88aa7fc700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 0x7f889c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.866+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f88b4083360 con 0x7f88b806df70 2023-06-23T18:04:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.866+0000 7f88bd27a700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 0x7f889c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.866+0000 7f88bd27a700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 0x7f889c0687d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b1710 tx=0x7f88ac0092e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:41.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:41.870+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f88b4050410 con 0x7f88b806df70 2023-06-23T18:04:42.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.034+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88b8158b60 con 0x7f88b806df70 2023-06-23T18:04:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.034+0000 7f88aa7fc700 1 -- 172.21.15.22:0/3306083861 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88b4017020 con 0x7f88b806df70 2023-06-23T18:04:42.049 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:42.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 msgr2=0x7f889c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 0x7f889c0687d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b1710 tx=0x7f88ac0092e0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 msgr2=0x7f88b80c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c410 tx=0x7f88b400c720 comp rx=0 tx=0).stop 2023-06-23T18:04:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 shutdown_connections 2023-06-23T18:04:42.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f889c066320 0x7f889c0687d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8159cd0 0x7f88b814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88b80b1f70 0x7f88b81efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 --2- 172.21.15.22:0/3306083861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88b806df70 0x7f88b80c51e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 >> 172.21.15.22:0/3306083861 conn(0x7f88b806b700 msgr2=0x7f88b80b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 shutdown_connections 2023-06-23T18:04:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.038+0000 7f88bea7d700 1 -- 172.21.15.22:0/3306083861 wait complete. 2023-06-23T18:04:42.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:42.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 Processor -- start 2023-06-23T18:04:42.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 -- start start 2023-06-23T18:04:42.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc524050760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc524036e80 0x7fc524053080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc524052bf0 con 0x7fc5240dfb70 2023-06-23T18:04:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240dd650 con 0x7fc524036e80 2023-06-23T18:04:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240528c0 con 0x7fc5240dd280 2023-06-23T18:04:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52a7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc5240dfb70 0x7fc524036770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51704/0 (socket says 172.21.15.22:51704) 2023-06-23T18:04:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52a7fc700 1 -- 172.21.15.22:0/74484185 learned_addr learned my addr 172.21.15.22:0/74484185 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52affd700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc524050760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/74484185 >> v2:172.21.15.169:3300/0 conn(0x7fc524036e80 0x7fc524053080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090479251 0 0) 0x7fc524052bf0 con 0x7fc5240dfb70 2023-06-23T18:04:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5180035d0 con 0x7fc5240dfb70 2023-06-23T18:04:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605219507 0 0) 0x7fc5180035d0 con 0x7fc5240dfb70 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 -- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 msgr2=0x7fc524050760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc524050760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 -- 172.21.15.22:0/74484185 >> v1:172.21.15.22:6789/0 conn(0x7fc5240dfb70 legacy=0x7fc524036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 -- 172.21.15.22:0/74484185 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240e0430 con 0x7fc524036e80 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52affd700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc524050760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.174+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc524053080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02c8d0 tx=0x7fc52c0674a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b861fc7170ab13e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c001670 con 0x7fc524036e80 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c04e4c0 con 0x7fc524036e80 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc529ffb700 1 -- 172.21.15.22:0/74484185 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c073a30 con 0x7fc524036e80 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 -- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 msgr2=0x7fc524053080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc524053080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02c8d0 tx=0x7fc52c0674a0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 -- 172.21.15.22:0/74484185 shutdown_connections 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc524053080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 --2- 172.21.15.22:0/74484185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc524050760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 -- 172.21.15.22:0/74484185 >> 172.21.15.22:0/74484185 conn(0x7fc524051770 msgr2=0x7fc524051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 -- 172.21.15.22:0/74484185 shutdown_connections 2023-06-23T18:04:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.178+0000 7fc52bfff700 1 -- 172.21.15.22:0/74484185 wait complete. 2023-06-23T18:04:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 Processor -- start 2023-06-23T18:04:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- start start 2023-06-23T18:04:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc5240d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc5240d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52404e0e0 con 0x7fc5240dfb70 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52404d4d0 con 0x7fc524036e80 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52404d740 con 0x7fc5240dd280 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46644/0 (socket says 172.21.15.22:46644) 2023-06-23T18:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 -- 172.21.15.22:0/3165331285 learned_addr learned my addr 172.21.15.22:0/3165331285 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52a7fc700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc5240d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52affd700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc5240d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 -- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 msgr2=0x7fc5240d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc5240d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 -- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 msgr2=0x7fc5240d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc5240d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 -- 172.21.15.22:0/3165331285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140094b0 con 0x7fc5240dfb70 2023-06-23T18:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52b7fe700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc52c068770 tx=0x7fc52c0683f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c066c90 con 0x7fc5240dfb70 2023-06-23T18:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c066df0 con 0x7fc5240dfb70 2023-06-23T18:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52404da20 con 0x7fc5240dfb70 2023-06-23T18:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c06d2e0 con 0x7fc5240dfb70 2023-06-23T18:04:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.182+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5240d4730 con 0x7fc5240dfb70 2023-06-23T18:04:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52404d880 con 0x7fc5240dfb70 2023-06-23T18:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc52c06d440 con 0x7fc5240dfb70 2023-06-23T18:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc5237fe700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 0x7fc50c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc52c0fda00 con 0x7fc5240dfb70 2023-06-23T18:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc52affd700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 0x7fc50c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.186+0000 7fc52affd700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 0x7fc50c0687a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc514009e00 tx=0x7fc514009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.194+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc52404d880 con 0x7fc5240dfb70 2023-06-23T18:04:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.358+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc52404e220 con 0x7fc5240dfb70 2023-06-23T18:04:42.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.358+0000 7fc5237fe700 1 -- 172.21.15.22:0/3165331285 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc52c09f020 con 0x7fc5240dfb70 2023-06-23T18:04:42.372 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 msgr2=0x7fc50c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 0x7fc50c0687a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc514009e00 tx=0x7fc514009240 comp rx=0 tx=0).stop 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 msgr2=0x7fc5240d94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc52c068770 tx=0x7fc52c0683f0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 shutdown_connections 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5240dfb70 0x7fc5240d94d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5240dd280 0x7fc5240d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc50c0662f0 0x7fc50c0687a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 --2- 172.21.15.22:0/3165331285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc524036e80 0x7fc5240d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 >> 172.21.15.22:0/3165331285 conn(0x7fc524051770 msgr2=0x7fc5240de560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 shutdown_connections 2023-06-23T18:04:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.362+0000 7fc52bfff700 1 -- 172.21.15.22:0/3165331285 wait complete. 2023-06-23T18:04:42.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:42.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 Processor -- start 2023-06-23T18:04:42.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 -- start start 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c14158b20 0x7f4c1406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1406eab0 con 0x7f4c14158ef0 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c140cd680 con 0x7f4c14070e50 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1b432700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c140cdf20 con 0x7f4c14158b20 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c19c2f700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4c14158ef0 0x7f4c14070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51712/0 (socket says 172.21.15.22:51712) 2023-06-23T18:04:42.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c19c2f700 1 -- 172.21.15.22:0/2676659088 learned_addr learned my addr 172.21.15.22:0/2676659088 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1a430700 1 --2- 172.21.15.22:0/2676659088 >> v2:172.21.15.169:3300/0 conn(0x7f4c14158b20 0x7f4c1406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1942e700 1 -- 172.21.15.22:0/2676659088 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148153602 0 0) 0x7f4c1406eab0 con 0x7f4c14158ef0 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1942e700 1 -- 172.21.15.22:0/2676659088 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c000035d0 con 0x7f4c14158ef0 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1a430700 1 -- 172.21.15.22:0/2676659088 >> v1:172.21.15.169:6789/0 conn(0x7f4c14070e50 legacy=0x7f4c140c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1a430700 1 -- 172.21.15.22:0/2676659088 >> v1:172.21.15.22:6789/0 conn(0x7f4c14158ef0 legacy=0x7f4c14070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.502+0000 7f4c1a430700 1 -- 172.21.15.22:0/2676659088 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140cd960 con 0x7f4c14158b20 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1a430700 1 --2- 172.21.15.22:0/2676659088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158b20 0x7f4c1406e440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009d90 tx=0x7f4c1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fafe7299c235c21c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1942e700 1 -- 172.21.15.22:0/2676659088 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000e070 con 0x7f4c14158b20 2023-06-23T18:04:42.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1942e700 1 -- 172.21.15.22:0/2676659088 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000bb20 con 0x7f4c14158b20 2023-06-23T18:04:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1942e700 1 -- 172.21.15.22:0/2676659088 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000be90 con 0x7f4c14158b20 2023-06-23T18:04:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- 172.21.15.22:0/2676659088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158b20 msgr2=0x7f4c1406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2676659088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158b20 0x7f4c1406e440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009d90 tx=0x7f4c1000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:42.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- 172.21.15.22:0/2676659088 shutdown_connections 2023-06-23T18:04:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2676659088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158b20 0x7f4c1406e440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- 172.21.15.22:0/2676659088 >> 172.21.15.22:0/2676659088 conn(0x7f4c1406b700 msgr2=0x7f4c140b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- 172.21.15.22:0/2676659088 shutdown_connections 2023-06-23T18:04:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- 172.21.15.22:0/2676659088 wait complete. 2023-06-23T18:04:42.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 Processor -- start 2023-06-23T18:04:42.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.506+0000 7f4c1b432700 1 -- start start 2023-06-23T18:04:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c14070e50 0x7f4c14149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158ef0 0x7f4c141574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141f2580 con 0x7f4c14158b20 2023-06-23T18:04:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141ee130 con 0x7f4c14158ef0 2023-06-23T18:04:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141495d0 con 0x7f4c14070e50 2023-06-23T18:04:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46650/0 (socket says 172.21.15.22:46650) 2023-06-23T18:04:42.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1a430700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c14070e50 0x7f4c14149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1ac31700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158ef0 0x7f4c141574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 -- 172.21.15.22:0/2851187906 learned_addr learned my addr 172.21.15.22:0/2851187906 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 -- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c14070e50 msgr2=0x7f4c14149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c14070e50 0x7f4c14149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 -- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158ef0 msgr2=0x7f4c141574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158ef0 0x7f4c141574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 -- 172.21.15.22:0/2851187906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009a70 con 0x7f4c14158b20 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c19c2f700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c4d0 tx=0x7f4c0400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04005d90 con 0x7f4c14158b20 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c040070d0 con 0x7f4c14158b20 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c141f1970 con 0x7f4c14158b20 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04007480 con 0x7f4c14158b20 2023-06-23T18:04:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.510+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c141f1f30 con 0x7f4c14158b20 2023-06-23T18:04:42.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c141f2730 con 0x7f4c14158b20 2023-06-23T18:04:42.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4c040075e0 con 0x7f4c14158b20 2023-06-23T18:04:42.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c0affd700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 0x7f4bf80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4c04083420 con 0x7f4c14158b20 2023-06-23T18:04:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c1a430700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 0x7f4bf80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.514+0000 7f4c1a430700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 0x7f4bf80687a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b370 tx=0x7f4c10011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.522+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4c141f2730 con 0x7f4c14158b20 2023-06-23T18:04:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.686+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c1404e4c0 con 0x7f4c14158b20 2023-06-23T18:04:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.686+0000 7f4c0affd700 1 -- 172.21.15.22:0/2851187906 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c04050580 con 0x7f4c14158b20 2023-06-23T18:04:42.699 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.686+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 msgr2=0x7f4bf80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 0x7f4bf80687a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b370 tx=0x7f4c10011000 comp rx=0 tx=0).stop 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 msgr2=0x7f4c14156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c4d0 tx=0x7f4c0400c7e0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 shutdown_connections 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f4bf80662f0 0x7f4bf80687a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c14158ef0 0x7f4c141574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c14158b20 0x7f4c14156f80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 --2- 172.21.15.22:0/2851187906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c14070e50 0x7f4c14149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 >> 172.21.15.22:0/2851187906 conn(0x7f4c1406b700 msgr2=0x7f4c140ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 shutdown_connections 2023-06-23T18:04:42.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.690+0000 7f4c1b432700 1 -- 172.21.15.22:0/2851187906 wait complete. 2023-06-23T18:04:42.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 Processor -- start 2023-06-23T18:04:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 -- start start 2023-06-23T18:04:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43c80cca20 0x7f43c80d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80d21d0 0x7f43c80d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c80d1d40 con 0x7f43c80b1920 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c80b1cf0 con 0x7f43c80cca20 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80cf130 con 0x7f43c80d21d0 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43c80cca20 0x7f43c80d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c659c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f43c80b1920 0x7f43c80cffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51720/0 (socket says 172.21.15.22:51720) 2023-06-23T18:04:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43c80cca20 0x7f43c80d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46662/0 (socket says 172.21.15.22:46662) 2023-06-23T18:04:42.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c659c700 1 -- 172.21.15.22:0/2432506736 learned_addr learned my addr 172.21.15.22:0/2432506736 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c6d9d700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80d21d0 0x7f43c80d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 -- 172.21.15.22:0/2432506736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80d21d0 msgr2=0x7f43c80d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80d21d0 0x7f43c80d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c559a700 1 -- 172.21.15.22:0/2432506736 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598217246 0 0) 0x7f43c80d1d40 con 0x7f43c80b1920 2023-06-23T18:04:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 -- 172.21.15.22:0/2432506736 >> v1:172.21.15.22:6789/0 conn(0x7f43c80b1920 legacy=0x7f43c80cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 -- 172.21.15.22:0/2432506736 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80cf410 con 0x7f43c80cca20 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80cca20 0x7f43c80d06b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b5dd6702df73b5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c559a700 1 -- 172.21.15.22:0/2432506736 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800e070 con 0x7f43c80cca20 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c559a700 1 -- 172.21.15.22:0/2432506736 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b800bbd0 con 0x7f43c80cca20 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.822+0000 7f43c559a700 1 -- 172.21.15.22:0/2432506736 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8012430 con 0x7f43c80cca20 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- 172.21.15.22:0/2432506736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80cca20 msgr2=0x7f43c80d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80cca20 0x7f43c80d06b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- 172.21.15.22:0/2432506736 shutdown_connections 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80d21d0 0x7f43c80d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- 172.21.15.22:0/2432506736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80cca20 0x7f43c80d06b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- 172.21.15.22:0/2432506736 >> 172.21.15.22:0/2432506736 conn(0x7f43c8069960 msgr2=0x7f43c8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- 172.21.15.22:0/2432506736 shutdown_connections 2023-06-23T18:04:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- 172.21.15.22:0/2432506736 wait complete. 2023-06-23T18:04:42.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 Processor -- start 2023-06-23T18:04:42.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- start start 2023-06-23T18:04:42.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 0x7f43c80c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80d21d0 0x7f43c80b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81f5020 con 0x7f43c80d21d0 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80ca900 con 0x7f43c80cca20 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80cabe0 con 0x7f43c80b1920 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.826+0000 7f43c659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:55946/0 (socket says 172.21.15.22:55946) 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 -- 172.21.15.22:0/1519626813 learned_addr learned my addr 172.21.15.22:0/1519626813 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 0x7f43c80c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c6d9d700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80d21d0 0x7f43c80b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 -- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 msgr2=0x7f43c80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 0x7f43c80c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 -- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80d21d0 msgr2=0x7f43c80b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80d21d0 0x7f43c80b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 -- 172.21.15.22:0/1519626813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009a70 con 0x7f43c80b1920 2023-06-23T18:04:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 0x7f43c80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c659c700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c580 tx=0x7f43b000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004ca0 con 0x7f43c80b1920 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0004e20 con 0x7f43c80b1920 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0007310 con 0x7f43c80b1920 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c80caec0 con 0x7f43c80b1920 2023-06-23T18:04:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c80cb480 con 0x7f43c80b1920 2023-06-23T18:04:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.830+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c815fef0 con 0x7f43c80b1920 2023-06-23T18:04:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.834+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f43b0007470 con 0x7f43c80b1920 2023-06-23T18:04:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.834+0000 7f43beffd700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 0x7f43a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.834+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 0x7f43a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.834+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f43b00842a0 con 0x7f43c80b1920 2023-06-23T18:04:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.834+0000 7f43c5d9b700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 0x7f43a80687a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b8011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:42.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:42.842+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f43b00514d0 con 0x7f43c80b1920 2023-06-23T18:04:43.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.002+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43c80cad20 con 0x7f43c80b1920 2023-06-23T18:04:43.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.002+0000 7f43beffd700 1 -- 172.21.15.22:0/1519626813 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43c80cad20 con 0x7f43c80b1920 2023-06-23T18:04:43.016 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 msgr2=0x7f43a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 0x7f43a80687a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b8011000 comp rx=0 tx=0).stop 2023-06-23T18:04:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 msgr2=0x7f43c81f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c580 tx=0x7f43b000c890 comp rx=0 tx=0).stop 2023-06-23T18:04:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 shutdown_connections 2023-06-23T18:04:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c80d21d0 0x7f43c80b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f43a80662f0 0x7f43a80687a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80cca20 0x7f43c80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 --2- 172.21.15.22:0/1519626813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80b1920 0x7f43c81f4ae0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 >> 172.21.15.22:0/1519626813 conn(0x7f43c8069960 msgr2=0x7f43c815f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 shutdown_connections 2023-06-23T18:04:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.006+0000 7f43c759e700 1 -- 172.21.15.22:0/1519626813 wait complete. 2023-06-23T18:04:43.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 Processor -- start 2023-06-23T18:04:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 -- start start 2023-06-23T18:04:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5400cf3a0 0x7fb5400cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 0x7fb5400ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400ccb30 con 0x7fb5400d1f30 2023-06-23T18:04:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5400cfcb0 con 0x7fb5400cf3a0 2023-06-23T18:04:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.134+0000 7fb546b36700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400d2880 con 0x7fb5400d0730 2023-06-23T18:04:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 0x7fb5400ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb545333700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb5400d1f30 0x7fb5400d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:59200/0 (socket says 172.21.15.22:59200) 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb545333700 1 -- 172.21.15.22:0/2853577240 learned_addr learned my addr 172.21.15.22:0/2853577240 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb545b34700 1 --2- 172.21.15.22:0/2853577240 >> v2:172.21.15.169:3300/0 conn(0x7fb5400cf3a0 0x7fb5400cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 -- 172.21.15.22:0/2853577240 >> v2:172.21.15.169:3300/0 conn(0x7fb5400cf3a0 msgr2=0x7fb5400cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 --2- 172.21.15.22:0/2853577240 >> v2:172.21.15.169:3300/0 conn(0x7fb5400cf3a0 0x7fb5400cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 -- 172.21.15.22:0/2853577240 >> v1:172.21.15.169:6789/0 conn(0x7fb5400d1f30 legacy=0x7fb5400d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 -- 172.21.15.22:0/2853577240 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400d2b60 con 0x7fb5400d0730 2023-06-23T18:04:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb544b32700 1 -- 172.21.15.22:0/2853577240 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597913792 0 0) 0x7fb5400ccb30 con 0x7fb5400d1f30 2023-06-23T18:04:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546335700 1 --2- 172.21.15.22:0/2853577240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 0x7fb5400ccfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb538009d90 tx=0x7fb53800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d188aac200e6097f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb544b32700 1 -- 172.21.15.22:0/2853577240 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53800e070 con 0x7fb5400d0730 2023-06-23T18:04:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb544b32700 1 -- 172.21.15.22:0/2853577240 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53800bb20 con 0x7fb5400d0730 2023-06-23T18:04:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb544b32700 1 -- 172.21.15.22:0/2853577240 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538012430 con 0x7fb5400d0730 2023-06-23T18:04:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 -- 172.21.15.22:0/2853577240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 msgr2=0x7fb5400ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 --2- 172.21.15.22:0/2853577240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 0x7fb5400ccfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb538009d90 tx=0x7fb53800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 -- 172.21.15.22:0/2853577240 shutdown_connections 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 --2- 172.21.15.22:0/2853577240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400d0730 0x7fb5400ccfc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 --2- 172.21.15.22:0/2853577240 >> v2:172.21.15.169:3300/0 conn(0x7fb5400cf3a0 0x7fb5400cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 -- 172.21.15.22:0/2853577240 >> 172.21.15.22:0/2853577240 conn(0x7fb54006a280 msgr2=0x7fb54009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 -- 172.21.15.22:0/2853577240 shutdown_connections 2023-06-23T18:04:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.138+0000 7fb546b36700 1 -- 172.21.15.22:0/2853577240 wait complete. 2023-06-23T18:04:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 Processor -- start 2023-06-23T18:04:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 -- start start 2023-06-23T18:04:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400cf3a0 0x7fb5400b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 0x7fb54014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540150760 con 0x7fb5400d0730 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54014a490 con 0x7fb5400d1f30 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546b36700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5401501e0 con 0x7fb5400cf3a0 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545333700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545333700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46678/0 (socket says 172.21.15.22:46678) 2023-06-23T18:04:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545333700 1 -- 172.21.15.22:0/2802426916 learned_addr learned my addr 172.21.15.22:0/2802426916 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb546335700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 0x7fb54014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545b34700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400cf3a0 0x7fb5400b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545333700 1 -- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400cf3a0 msgr2=0x7fb5400b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.142+0000 7fb545333700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400cf3a0 0x7fb5400b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb545333700 1 -- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 msgr2=0x7fb54014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb545333700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 0x7fb54014c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb545333700 1 -- 172.21.15.22:0/2802426916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538009a70 con 0x7fb5400d0730 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb546335700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 0x7fb54014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb545333700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c4e0 tx=0x7fb53000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530004ca0 con 0x7fb5400d0730 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530004e20 con 0x7fb5400d0730 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5400b7c40 con 0x7fb5400d0730 2023-06-23T18:04:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530007310 con 0x7fb5400d0730 2023-06-23T18:04:43.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5400b7680 con 0x7fb5400d0730 2023-06-23T18:04:43.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.146+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54014a5d0 con 0x7fb5400d0730 2023-06-23T18:04:43.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.150+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb530007470 con 0x7fb5400d0730 2023-06-23T18:04:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.150+0000 7fb5367fc700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 0x7fb5240687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.150+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb530083c90 con 0x7fb5400d0730 2023-06-23T18:04:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.150+0000 7fb545b34700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 0x7fb5240687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.150+0000 7fb545b34700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 0x7fb5240687a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009b70 tx=0x7fb53c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.158+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb530050e70 con 0x7fb5400d0730 2023-06-23T18:04:43.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.314+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb54004e4c0 con 0x7fb5400d0730 2023-06-23T18:04:43.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.314+0000 7fb5367fc700 1 -- 172.21.15.22:0/2802426916 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb530007720 con 0x7fb5400d0730 2023-06-23T18:04:43.328 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:43.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 msgr2=0x7fb5240687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 0x7fb5240687a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009b70 tx=0x7fb53c008040 comp rx=0 tx=0).stop 2023-06-23T18:04:43.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 msgr2=0x7fb54014fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c4e0 tx=0x7fb53000c8a0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 shutdown_connections 2023-06-23T18:04:43.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb5240662f0 0x7fb5240687a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5400d1f30 0x7fb54014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5400d0730 0x7fb54014fca0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 --2- 172.21.15.22:0/2802426916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5400cf3a0 0x7fb5400b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 >> 172.21.15.22:0/2802426916 conn(0x7fb54006a280 msgr2=0x7fb5400b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 shutdown_connections 2023-06-23T18:04:43.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.318+0000 7fb546b36700 1 -- 172.21.15.22:0/2802426916 wait complete. 2023-06-23T18:04:43.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:43.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 Processor -- start 2023-06-23T18:04:43.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 -- start start 2023-06-23T18:04:43.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 0x7f63200afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63200b0420 con 0x7f6320071450 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632006d0d0 con 0x7f63200affd0 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6325469700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632006d560 con 0x7f6320159c30 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6324c68700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6320071450 0x7f63200cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51736/0 (socket says 172.21.15.22:51736) 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f6324c68700 1 -- 172.21.15.22:0/1647520993 learned_addr learned my addr 172.21.15.22:0/1647520993 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631ffff700 1 --2- 172.21.15.22:0/1647520993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 0x7f63200afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740652462 0 0) 0x7f6318001110 con 0x7f6320071450 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63080035d0 con 0x7f6320071450 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396351151 0 0) 0x7f632006d0d0 con 0x7f63200affd0 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6318001110 con 0x7f63200affd0 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631ffff700 1 -- 172.21.15.22:0/1647520993 >> v1:172.21.15.169:6789/0 conn(0x7f63200affd0 legacy=0x7f6320070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631ffff700 1 -- 172.21.15.22:0/1647520993 >> v1:172.21.15.22:6789/0 conn(0x7f6320071450 legacy=0x7f63200cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631ffff700 1 -- 172.21.15.22:0/1647520993 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63200718d0 con 0x7f6320159c30 2023-06-23T18:04:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.454+0000 7f631ffff700 1 --2- 172.21.15.22:0/1647520993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 0x7f63200afa90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314009880 tx=0x7f6314009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747003c5fa5128ea server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314011830 con 0x7f6320159c30 2023-06-23T18:04:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314011990 con 0x7f6320159c30 2023-06-23T18:04:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f631effd700 1 -- 172.21.15.22:0/1647520993 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314011d20 con 0x7f6320159c30 2023-06-23T18:04:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- 172.21.15.22:0/1647520993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 msgr2=0x7f63200afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 --2- 172.21.15.22:0/1647520993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 0x7f63200afa90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314009880 tx=0x7f6314009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- 172.21.15.22:0/1647520993 shutdown_connections 2023-06-23T18:04:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 --2- 172.21.15.22:0/1647520993 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6320159c30 0x7f63200afa90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- 172.21.15.22:0/1647520993 >> 172.21.15.22:0/1647520993 conn(0x7f632006a5f0 msgr2=0x7f63200caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- 172.21.15.22:0/1647520993 shutdown_connections 2023-06-23T18:04:43.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- 172.21.15.22:0/1647520993 wait complete. 2023-06-23T18:04:43.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 Processor -- start 2023-06-23T18:04:43.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.458+0000 7f6325469700 1 -- start start 2023-06-23T18:04:43.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 0x7f63201548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63200affd0 0x7f63201f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63200c4120 con 0x7f6320159c30 2023-06-23T18:04:43.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6320154df0 con 0x7f6320071450 2023-06-23T18:04:43.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63201f1cc0 con 0x7f63200affd0 2023-06-23T18:04:43.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6324c68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6324c68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46682/0 (socket says 172.21.15.22:46682) 2023-06-23T18:04:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 0x7f63201548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63200affd0 0x7f63201f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6324c68700 1 -- 172.21.15.22:0/3016712558 learned_addr learned my addr 172.21.15.22:0/3016712558 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 -- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63200affd0 msgr2=0x7f63201f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63200affd0 0x7f63201f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 -- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 msgr2=0x7f63200c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 -- 172.21.15.22:0/3016712558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140094b0 con 0x7f6320071450 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6324c68700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631ffff700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 0x7f63201548b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6314015000 tx=0x7f6314004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401a8c0 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631401e070 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63201f1fa0 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401ac80 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.462+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63201f1960 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632004e4c0 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f63140233b0 con 0x7f6320071450 2023-06-23T18:04:43.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f631cff9700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 0x7f630c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f631f7fe700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 0x7f630c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6314084f30 con 0x7f6320071450 2023-06-23T18:04:43.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.466+0000 7f631f7fe700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 0x7f630c0687a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6310001560 tx=0x7f6310009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.474+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f6314052090 con 0x7f6320071450 2023-06-23T18:04:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.654+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63200b3f90 con 0x7f6320071450 2023-06-23T18:04:43.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.654+0000 7f631cff9700 1 -- 172.21.15.22:0/3016712558 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f63140556b0 con 0x7f6320071450 2023-06-23T18:04:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 msgr2=0x7f630c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 0x7f630c0687a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6310001560 tx=0x7f6310009040 comp rx=0 tx=0).stop 2023-06-23T18:04:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 msgr2=0x7f63201548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 0x7f63201548b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6314015000 tx=0x7f6314004770 comp rx=0 tx=0).stop 2023-06-23T18:04:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 shutdown_connections 2023-06-23T18:04:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6320159c30 0x7f63200c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63200affd0 0x7f63201f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f630c0662f0 0x7f630c0687a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 --2- 172.21.15.22:0/3016712558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6320071450 0x7f63201548b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 >> 172.21.15.22:0/3016712558 conn(0x7f632006a5f0 msgr2=0x7f6320070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 shutdown_connections 2023-06-23T18:04:43.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.658+0000 7f6325469700 1 -- 172.21.15.22:0/3016712558 wait complete. 2023-06-23T18:04:43.674 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:43.682 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2023-06-23T18:04:43.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:43.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-06-23T18:04:43.697 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:43.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:04:43.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:43.698 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-23T18:04:43.704 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:43.704 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:43.712 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:43.712 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:43.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.850+0000 7f938f59e700 1 Processor -- start 2023-06-23T18:04:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938f59e700 1 -- start start 2023-06-23T18:04:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 0x7f93900ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938f59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900b7ec0 con 0x7f93900cf480 2023-06-23T18:04:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900b1ad0 con 0x7f93900b7af0 2023-06-23T18:04:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93900d0a10 con 0x7f93900cc9e0 2023-06-23T18:04:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938dd9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f93900cf480 0x7f93900d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51738/0 (socket says 172.21.15.22:51738) 2023-06-23T18:04:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938dd9b700 1 -- 172.21.15.22:0/1156588889 learned_addr learned my addr 172.21.15.22:0/1156588889 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938ed9d700 1 --2- 172.21.15.22:0/1156588889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 0x7f93900ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554841403 0 0) 0x7f93900b7ec0 con 0x7f93900cf480 2023-06-23T18:04:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93780035d0 con 0x7f93900cf480 2023-06-23T18:04:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797678848 0 0) 0x7f93780035d0 con 0x7f93900cf480 2023-06-23T18:04:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938ed9d700 1 -- 172.21.15.22:0/1156588889 >> v1:172.21.15.169:6789/0 conn(0x7f93900b7af0 legacy=0x7f93900ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938ed9d700 1 -- 172.21.15.22:0/1156588889 >> v1:172.21.15.22:6789/0 conn(0x7f93900cf480 legacy=0x7f93900d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938ed9d700 1 -- 172.21.15.22:0/1156588889 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900d0cb0 con 0x7f93900cc9e0 2023-06-23T18:04:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938ed9d700 1 --2- 172.21.15.22:0/1156588889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 0x7f93900ce540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9388009dc0 tx=0x7f938800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e00f400694ddc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938800e070 con 0x7f93900cc9e0 2023-06-23T18:04:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9388003a40 con 0x7f93900cc9e0 2023-06-23T18:04:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.854+0000 7f938d59a700 1 -- 172.21.15.22:0/1156588889 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380001670 con 0x7f93900cc9e0 2023-06-23T18:04:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- 172.21.15.22:0/1156588889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 msgr2=0x7f93900ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 --2- 172.21.15.22:0/1156588889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 0x7f93900ce540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9388009dc0 tx=0x7f938800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- 172.21.15.22:0/1156588889 shutdown_connections 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 --2- 172.21.15.22:0/1156588889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900cc9e0 0x7f93900ce540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- 172.21.15.22:0/1156588889 >> 172.21.15.22:0/1156588889 conn(0x7f9390069960 msgr2=0x7f9390069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- 172.21.15.22:0/1156588889 shutdown_connections 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- 172.21.15.22:0/1156588889 wait complete. 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 Processor -- start 2023-06-23T18:04:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- start start 2023-06-23T18:04:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900b7af0 0x7f939015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93900cf480 0x7f93900b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939015d760 con 0x7f93900cc9e0 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939015cbe0 con 0x7f93900cf480 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.858+0000 7f938f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939015cec0 con 0x7f93900b7af0 2023-06-23T18:04:43.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46690/0 (socket says 172.21.15.22:46690) 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 -- 172.21.15.22:0/3358637152 learned_addr learned my addr 172.21.15.22:0/3358637152 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938e59c700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900b7af0 0x7f939015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938ed9d700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93900cf480 0x7f93900b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 -- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900b7af0 msgr2=0x7f939015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900b7af0 0x7f939015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 -- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93900cf480 msgr2=0x7f93900b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:43.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93900cf480 0x7f93900b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:43.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 -- 172.21.15.22:0/3358637152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388009a70 con 0x7f93900cc9e0 2023-06-23T18:04:43.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938dd9b700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f938000c700 tx=0x7f938000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380004ca0 con 0x7f93900cc9e0 2023-06-23T18:04:43.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9380004e00 con 0x7f93900cc9e0 2023-06-23T18:04:43.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939015d1a0 con 0x7f93900cc9e0 2023-06-23T18:04:43.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380007310 con 0x7f93900cc9e0 2023-06-23T18:04:43.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93900c7e90 con 0x7f93900cc9e0 2023-06-23T18:04:43.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.862+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900c80b0 con 0x7f93900cc9e0 2023-06-23T18:04:43.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.866+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9380007470 con 0x7f93900cc9e0 2023-06-23T18:04:43.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.866+0000 7f937effd700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 0x7f93700687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:43.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.866+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9380083c90 con 0x7f93900cc9e0 2023-06-23T18:04:43.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.866+0000 7f938e59c700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 0x7f93700687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:43.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.866+0000 7f938e59c700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 0x7f93700687d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93900c5ac0 tx=0x7f9384009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:43.874+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f93900c80b0 con 0x7f93900cc9e0 2023-06-23T18:04:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.042+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93900c80b0 con 0x7f93900cc9e0 2023-06-23T18:04:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.046+0000 7f937effd700 1 -- 172.21.15.22:0/3358637152 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f93900c80b0 con 0x7f93900cc9e0 2023-06-23T18:04:44.059 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:44.060 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 msgr2=0x7f93700687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 0x7f93700687d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93900c5ac0 tx=0x7f9384009040 comp rx=0 tx=0).stop 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 msgr2=0x7f93900b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f938000c700 tx=0x7f938000ca10 comp rx=0 tx=0).stop 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 shutdown_connections 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9370066320 0x7f93700687d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93900cf480 0x7f93900b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93900cc9e0 0x7f93900b6480 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 --2- 172.21.15.22:0/3358637152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93900b7af0 0x7f939015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 >> 172.21.15.22:0/3358637152 conn(0x7f9390069960 msgr2=0x7f93900723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 shutdown_connections 2023-06-23T18:04:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.050+0000 7f938f59e700 1 -- 172.21.15.22:0/3358637152 wait complete. 2023-06-23T18:04:44.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:44.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.182+0000 7f873b59e700 1 Processor -- start 2023-06-23T18:04:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.182+0000 7f873b59e700 1 -- start start 2023-06-23T18:04:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f873c070e50 0x7f873c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c06eab0 con 0x7f873c158b20 2023-06-23T18:04:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c0cdf20 con 0x7f873c070e50 2023-06-23T18:04:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0cd680 con 0x7f873c158ef0 2023-06-23T18:04:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873a59c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f873c158b20 0x7f873c070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51748/0 (socket says 172.21.15.22:51748) 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873a59c700 1 -- 172.21.15.22:0/3519705948 learned_addr learned my addr 172.21.15.22:0/3519705948 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 --2- 172.21.15.22:0/3519705948 >> v2:172.21.15.169:3300/0 conn(0x7f873c070e50 0x7f873c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f8739d9b700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873959a700 1 -- 172.21.15.22:0/3519705948 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932433698 0 0) 0x7f873c06eab0 con 0x7f873c158b20 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873959a700 1 -- 172.21.15.22:0/3519705948 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87240035d0 con 0x7f873c158b20 2023-06-23T18:04:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 -- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 msgr2=0x7f873c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 -- 172.21.15.22:0/3519705948 >> v1:172.21.15.22:6789/0 conn(0x7f873c158b20 legacy=0x7f873c070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 -- 172.21.15.22:0/3519705948 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0cd960 con 0x7f873c070e50 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873ad9d700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c0b1780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8734009d90 tx=0x7f873400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df5f204c3f24de5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873959a700 1 -- 172.21.15.22:0/3519705948 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873400e070 con 0x7f873c070e50 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873959a700 1 -- 172.21.15.22:0/3519705948 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873400bb20 con 0x7f873c070e50 2023-06-23T18:04:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873959a700 1 -- 172.21.15.22:0/3519705948 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873400be90 con 0x7f873c070e50 2023-06-23T18:04:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 msgr2=0x7f873c0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c0b1780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8734009d90 tx=0x7f873400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- 172.21.15.22:0/3519705948 shutdown_connections 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c0b1780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 --2- 172.21.15.22:0/3519705948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- 172.21.15.22:0/3519705948 >> 172.21.15.22:0/3519705948 conn(0x7f873c06b700 msgr2=0x7f873c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- 172.21.15.22:0/3519705948 shutdown_connections 2023-06-23T18:04:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.186+0000 7f873b59e700 1 -- 172.21.15.22:0/3519705948 wait complete. 2023-06-23T18:04:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 Processor -- start 2023-06-23T18:04:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 -- start start 2023-06-23T18:04:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c0c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1effb0 con 0x7f873c158b20 2023-06-23T18:04:44.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0c4280 con 0x7f873c070e50 2023-06-23T18:04:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0c4560 con 0x7f873c158ef0 2023-06-23T18:04:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46694/0 (socket says 172.21.15.22:46694) 2023-06-23T18:04:44.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 -- 172.21.15.22:0/3628752315 learned_addr learned my addr 172.21.15.22:0/3628752315 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:44.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873ad9d700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c0c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f873a59c700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 -- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 msgr2=0x7f873c0c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c0c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 -- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 msgr2=0x7f873c152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 -- 172.21.15.22:0/3628752315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734009a70 con 0x7f873c158b20 2023-06-23T18:04:44.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.190+0000 7f8739d9b700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f873000e840 tx=0x7f873000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300095a0 con 0x7f873c158b20 2023-06-23T18:04:44.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730013680 con 0x7f873c158b20 2023-06-23T18:04:44.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c0c4840 con 0x7f873c158b20 2023-06-23T18:04:44.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300139f0 con 0x7f873c158b20 2023-06-23T18:04:44.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c0c4e00 con 0x7f873c158b20 2023-06-23T18:04:44.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c04e4c0 con 0x7f873c158b20 2023-06-23T18:04:44.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.194+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8730013b50 con 0x7f873c158b20 2023-06-23T18:04:44.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.198+0000 7f872affd700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 0x7f871c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.198+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f87300845e0 con 0x7f873c158b20 2023-06-23T18:04:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.198+0000 7f873a59c700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 0x7f871c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.198+0000 7f873a59c700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 0x7f871c0687d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f872c007fd0 tx=0x7f872c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.202+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8730055250 con 0x7f873c158b20 2023-06-23T18:04:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.366+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f873c0b1aa0 con 0x7f873c158b20 2023-06-23T18:04:44.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.366+0000 7f872affd700 1 -- 172.21.15.22:0/3628752315 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8730054de0 con 0x7f873c158b20 2023-06-23T18:04:44.380 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 msgr2=0x7f871c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 0x7f871c0687d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f872c007fd0 tx=0x7f872c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 msgr2=0x7f873c1efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f873000e840 tx=0x7f873000eb50 comp rx=0 tx=0).stop 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 shutdown_connections 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f873c158ef0 0x7f873c0c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f873c158b20 0x7f873c1efa70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f871c066320 0x7f871c0687d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 --2- 172.21.15.22:0/3628752315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f873c070e50 0x7f873c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 >> 172.21.15.22:0/3628752315 conn(0x7f873c06b700 msgr2=0x7f873c0724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 shutdown_connections 2023-06-23T18:04:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.370+0000 7f873b59e700 1 -- 172.21.15.22:0/3628752315 wait complete. 2023-06-23T18:04:44.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 Processor -- start 2023-06-23T18:04:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 -- start start 2023-06-23T18:04:44.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79c40cf3a0 0x7f79c40cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79c40d0730 0x7f79c40ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c40ccb30 con 0x7f79c40d1f30 2023-06-23T18:04:44.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40cfcb0 con 0x7f79c40cf3a0 2023-06-23T18:04:44.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.510+0000 7f79cad59700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40d2880 con 0x7f79c40d0730 2023-06-23T18:04:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79c40cf3a0 0x7f79c40cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9556700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f79c40d1f30 0x7f79c40d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51762/0 (socket says 172.21.15.22:51762) 2023-06-23T18:04:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9556700 1 -- 172.21.15.22:0/2788190309 learned_addr learned my addr 172.21.15.22:0/2788190309 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79ca558700 1 --2- 172.21.15.22:0/2788190309 >> v2:172.21.15.169:3300/0 conn(0x7f79c40d0730 0x7f79c40ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 -- 172.21.15.22:0/2788190309 >> v2:172.21.15.169:3300/0 conn(0x7f79c40d0730 msgr2=0x7f79c40ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 --2- 172.21.15.22:0/2788190309 >> v2:172.21.15.169:3300/0 conn(0x7f79c40d0730 0x7f79c40ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 -- 172.21.15.22:0/2788190309 >> v1:172.21.15.22:6789/0 conn(0x7f79c40d1f30 legacy=0x7f79c40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c8d55700 1 -- 172.21.15.22:0/2788190309 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712194268 0 0) 0x7f79c40ccb30 con 0x7f79c40d1f30 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 -- 172.21.15.22:0/2788190309 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40d2b60 con 0x7f79c40cf3a0 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c9d57700 1 --2- 172.21.15.22:0/2788190309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40cf3a0 0x7f79c40cf770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f79c40ceb00 tx=0x7f79c0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761e2c0c9544e284 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c8d55700 1 -- 172.21.15.22:0/2788190309 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011830 con 0x7f79c40cf3a0 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c8d55700 1 -- 172.21.15.22:0/2788190309 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c0011990 con 0x7f79c40cf3a0 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79c8d55700 1 -- 172.21.15.22:0/2788190309 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011d20 con 0x7f79c40cf3a0 2023-06-23T18:04:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 -- 172.21.15.22:0/2788190309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40cf3a0 msgr2=0x7f79c40cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 --2- 172.21.15.22:0/2788190309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40cf3a0 0x7f79c40cf770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f79c40ceb00 tx=0x7f79c0009b20 comp rx=0 tx=0).stop 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 -- 172.21.15.22:0/2788190309 shutdown_connections 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 --2- 172.21.15.22:0/2788190309 >> v2:172.21.15.169:3300/0 conn(0x7f79c40d0730 0x7f79c40ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 --2- 172.21.15.22:0/2788190309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40cf3a0 0x7f79c40cf770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 -- 172.21.15.22:0/2788190309 >> 172.21.15.22:0/2788190309 conn(0x7f79c406a280 msgr2=0x7f79c409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 -- 172.21.15.22:0/2788190309 shutdown_connections 2023-06-23T18:04:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.514+0000 7f79cad59700 1 -- 172.21.15.22:0/2788190309 wait complete. 2023-06-23T18:04:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 Processor -- start 2023-06-23T18:04:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- start start 2023-06-23T18:04:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40cf3a0 0x7f79c40c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d1f30 0x7f79c415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c415c4b0 con 0x7f79c40d0730 2023-06-23T18:04:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c415c700 con 0x7f79c40d1f30 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c415c9e0 con 0x7f79c40cf3a0 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46714/0 (socket says 172.21.15.22:46714) 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 -- 172.21.15.22:0/2709702620 learned_addr learned my addr 172.21.15.22:0/2709702620 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9d57700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40cf3a0 0x7f79c40c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79ca558700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d1f30 0x7f79c415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 -- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40cf3a0 msgr2=0x7f79c40c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40cf3a0 0x7f79c40c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 -- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d1f30 msgr2=0x7f79c415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d1f30 0x7f79c415ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 -- 172.21.15.22:0/2709702620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0009450 con 0x7f79c40d0730 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79c9556700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c4e0 tx=0x7f79b400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004ca0 con 0x7f79c40d0730 2023-06-23T18:04:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4004e20 con 0x7f79c40d0730 2023-06-23T18:04:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c40cc410 con 0x7f79c40d0730 2023-06-23T18:04:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4007310 con 0x7f79c40d0730 2023-06-23T18:04:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.518+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c415b430 con 0x7f79c40d0730 2023-06-23T18:04:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c40cc930 con 0x7f79c40d0730 2023-06-23T18:04:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f79b4007470 con 0x7f79c40d0730 2023-06-23T18:04:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79ba7fc700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 0x7f79a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79c9d57700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 0x7f79a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f79b4083c50 con 0x7f79c40d0730 2023-06-23T18:04:44.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.522+0000 7f79c9d57700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 0x7f79a80687a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79c00090b0 tx=0x7f79c0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.530+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f79c40cc930 con 0x7f79c40d0730 2023-06-23T18:04:44.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79c40cc930 con 0x7f79c40d0730 2023-06-23T18:04:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79ba7fc700 1 -- 172.21.15.22:0/2709702620 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79c40cc930 con 0x7f79c40d0730 2023-06-23T18:04:44.706 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:44.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 msgr2=0x7f79a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 0x7f79a80687a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79c00090b0 tx=0x7f79c0008030 comp rx=0 tx=0).stop 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 msgr2=0x7f79c40c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c4e0 tx=0x7f79b400c7f0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 shutdown_connections 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f79a80662f0 0x7f79a80687a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d1f30 0x7f79c415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c40d0730 0x7f79c40c9520 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 --2- 172.21.15.22:0/2709702620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40cf3a0 0x7f79c40c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 >> 172.21.15.22:0/2709702620 conn(0x7f79c406a280 msgr2=0x7f79c40b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 shutdown_connections 2023-06-23T18:04:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.694+0000 7f79cad59700 1 -- 172.21.15.22:0/2709702620 wait complete. 2023-06-23T18:04:44.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:44.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 Processor -- start 2023-06-23T18:04:44.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 -- start start 2023-06-23T18:04:44.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc15bf10 0x7fbcdc06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbcdc06c510 0x7fbcdc0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc0cddf0 con 0x7fbcdc06f660 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc06cf70 con 0x7fbcdc06c510 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce2e4d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc06c9b0 con 0x7fbcdc15bf10 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbcdc06c510 0x7fbcdc0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbcdc06c510 0x7fbcdc0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46728/0 (socket says 172.21.15.22:46728) 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 -- 172.21.15.22:0/244003203 learned_addr learned my addr 172.21.15.22:0/244003203 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce1e4b700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc15bf10 0x7fbcdc06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 -- 172.21.15.22:0/244003203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc15bf10 msgr2=0x7fbcdc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc15bf10 0x7fbcdc06bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 -- 172.21.15.22:0/244003203 >> v1:172.21.15.169:6789/0 conn(0x7fbcdc06f660 legacy=0x7fbcdc15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 -- 172.21.15.22:0/244003203 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc06cc90 con 0x7fbcdc06c510 2023-06-23T18:04:44.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce164a700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc0c89f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f339372c95cc97d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce0e49700 1 -- 172.21.15.22:0/244003203 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800e070 con 0x7fbcdc06c510 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce0e49700 1 -- 172.21.15.22:0/244003203 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd800b9c0 con 0x7fbcdc06c510 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.830+0000 7fbce0e49700 1 -- 172.21.15.22:0/244003203 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800bd70 con 0x7fbcdc06c510 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- 172.21.15.22:0/244003203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 msgr2=0x7fbcdc0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc0c89f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- 172.21.15.22:0/244003203 shutdown_connections 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc0c89f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/244003203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc15bf10 0x7fbcdc06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- 172.21.15.22:0/244003203 >> 172.21.15.22:0/244003203 conn(0x7fbcdc06a5f0 msgr2=0x7fbcdc0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- 172.21.15.22:0/244003203 shutdown_connections 2023-06-23T18:04:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- 172.21.15.22:0/244003203 wait complete. 2023-06-23T18:04:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 Processor -- start 2023-06-23T18:04:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.834+0000 7fbce2e4d700 1 -- start start 2023-06-23T18:04:44.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc06f660 0x7fbcdc14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcdc15bf10 0x7fbcdc09b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc14af80 con 0x7fbcdc06c510 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc151eb0 con 0x7fbcdc15bf10 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc155340 con 0x7fbcdc06f660 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46744/0 (socket says 172.21.15.22:46744) 2023-06-23T18:04:44.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 -- 172.21.15.22:0/2713091925 learned_addr learned my addr 172.21.15.22:0/2713091925 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:44.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce264c700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcdc15bf10 0x7fbcdc09b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce164a700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc06f660 0x7fbcdc14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 -- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc06f660 msgr2=0x7fbcdc14a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc06f660 0x7fbcdc14a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 -- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcdc15bf10 msgr2=0x7fbcdc09b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcdc15bf10 0x7fbcdc09b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 -- 172.21.15.22:0/2713091925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009b30 con 0x7fbcdc06c510 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce1e4b700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e810 tx=0x7fbccc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc013610 con 0x7fbcdc06c510 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc013770 con 0x7fbcdc06c510 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc013b20 con 0x7fbcdc06c510 2023-06-23T18:04:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc14a9c0 con 0x7fbcdc06c510 2023-06-23T18:04:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc1555e0 con 0x7fbcdc06c510 2023-06-23T18:04:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.838+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc1551a0 con 0x7fbcdc06c510 2023-06-23T18:04:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.842+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fbccc013c80 con 0x7fbcdc06c510 2023-06-23T18:04:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.842+0000 7fbcd27fc700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 0x7fbcc00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.842+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbccc083ce0 con 0x7fbcdc06c510 2023-06-23T18:04:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.842+0000 7fbce164a700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 0x7fbcc00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.842+0000 7fbce164a700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 0x7fbcc00687d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc06c8e0 tx=0x7fbcd8009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:44.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:44.850+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fbcdc1551a0 con 0x7fbcdc06c510 2023-06-23T18:04:45.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.014+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcdc1551a0 con 0x7fbcdc06c510 2023-06-23T18:04:45.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.014+0000 7fbcd27fc700 1 -- 172.21.15.22:0/2713091925 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcdc1551a0 con 0x7fbcdc06c510 2023-06-23T18:04:45.028 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 msgr2=0x7fbcc00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 0x7fbcc00687d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc06c8e0 tx=0x7fbcd8009000 comp rx=0 tx=0).stop 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 msgr2=0x7fbcdc148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e810 tx=0x7fbccc00eb20 comp rx=0 tx=0).stop 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 shutdown_connections 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fbcc0066320 0x7fbcc00687d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcdc15bf10 0x7fbcdc09b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcdc06f660 0x7fbcdc14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 --2- 172.21.15.22:0/2713091925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcdc06c510 0x7fbcdc148400 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 >> 172.21.15.22:0/2713091925 conn(0x7fbcdc06a5f0 msgr2=0x7fbcdc0ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 shutdown_connections 2023-06-23T18:04:45.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.018+0000 7fbce2e4d700 1 -- 172.21.15.22:0/2713091925 wait complete. 2023-06-23T18:04:45.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 Processor -- start 2023-06-23T18:04:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 -- start start 2023-06-23T18:04:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81fc0cf3a0 0x7f81fc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0ccb30 con 0x7f81fc0d1f30 2023-06-23T18:04:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0cfcb0 con 0x7f81fc0d0730 2023-06-23T18:04:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8201fd5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0d2880 con 0x7f81fc0cf3a0 2023-06-23T18:04:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f81fbfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f81fc0d1f30 0x7f81fc0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51770/0 (socket says 172.21.15.22:51770) 2023-06-23T18:04:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f81fbfff700 1 -- 172.21.15.22:0/120303645 learned_addr learned my addr 172.21.15.22:0/120303645 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.154+0000 7f8200fd3700 1 --2- 172.21.15.22:0/120303645 >> v2:172.21.15.169:3300/0 conn(0x7f81fc0cf3a0 0x7f81fc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800532965 0 0) 0x7f81fc0ccb30 con 0x7f81fc0d1f30 2023-06-23T18:04:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e40035d0 con 0x7f81fc0d1f30 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8200fd3700 1 -- 172.21.15.22:0/120303645 >> v1:172.21.15.169:6789/0 conn(0x7f81fc0d0730 legacy=0x7f81fc0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642657308 0 0) 0x7f81e40035d0 con 0x7f81fc0d1f30 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8200fd3700 1 -- 172.21.15.22:0/120303645 >> v1:172.21.15.22:6789/0 conn(0x7f81fc0d1f30 legacy=0x7f81fc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8200fd3700 1 -- 172.21.15.22:0/120303645 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0d2b60 con 0x7f81fc0cf3a0 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8200fd3700 1 --2- 172.21.15.22:0/120303645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0cf3a0 0x7f81fc0cf770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009840 tx=0x7f81f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3b2527f91deac6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec001670 con 0x7f81fc0cf3a0 2023-06-23T18:04:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f0004030 con 0x7f81fc0cf3a0 2023-06-23T18:04:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f81fb7fe700 1 -- 172.21.15.22:0/120303645 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f00043c0 con 0x7f81fc0cf3a0 2023-06-23T18:04:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- 172.21.15.22:0/120303645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0cf3a0 msgr2=0x7f81fc0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 --2- 172.21.15.22:0/120303645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0cf3a0 0x7f81fc0cf770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009840 tx=0x7f81f0009b50 comp rx=0 tx=0).stop 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- 172.21.15.22:0/120303645 shutdown_connections 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 --2- 172.21.15.22:0/120303645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0cf3a0 0x7f81fc0cf770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- 172.21.15.22:0/120303645 >> 172.21.15.22:0/120303645 conn(0x7f81fc06a280 msgr2=0x7f81fc09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- 172.21.15.22:0/120303645 shutdown_connections 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- 172.21.15.22:0/120303645 wait complete. 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 Processor -- start 2023-06-23T18:04:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.158+0000 7f8201fd5700 1 -- start start 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0d0730 0x7f81fc1f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81fc0d1f30 0x7f81fc09fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0b6400 con 0x7f81fc0b5280 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0b5bf0 con 0x7f81fc0d0730 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0b2ca0 con 0x7f81fc0d1f30 2023-06-23T18:04:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46752/0 (socket says 172.21.15.22:46752) 2023-06-23T18:04:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 -- 172.21.15.22:0/4145183010 learned_addr learned my addr 172.21.15.22:0/4145183010 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8200fd3700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0d0730 0x7f81fc1f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f81fbfff700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81fc0d1f30 0x7f81fc09fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 -- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81fc0d1f30 msgr2=0x7f81fc09fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81fc0d1f30 0x7f81fc09fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 -- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0d0730 msgr2=0x7f81fc1f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0d0730 0x7f81fc1f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 -- 172.21.15.22:0/4145183010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0009450 con 0x7f81fc0b5280 2023-06-23T18:04:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f82017d4700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f81f400cb90 tx=0x7f81f400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003c20 con 0x7f81fc0b5280 2023-06-23T18:04:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4003d80 con 0x7f81fc0b5280 2023-06-23T18:04:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc0b5e40 con 0x7f81fc0b5280 2023-06-23T18:04:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400aae0 con 0x7f81fc0b5280 2023-06-23T18:04:45.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc0b7ae0 con 0x7f81fc0b5280 2023-06-23T18:04:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.162+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc0d2320 con 0x7f81fc0b5280 2023-06-23T18:04:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.166+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f81f400ac40 con 0x7f81fc0b5280 2023-06-23T18:04:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.166+0000 7f81f97fa700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 0x7f81e80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.166+0000 7f8200fd3700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 0x7f81e80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.166+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f81f4083f50 con 0x7f81fc0b5280 2023-06-23T18:04:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.166+0000 7f8200fd3700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 0x7f81e80687a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81fc08bc90 tx=0x7f81f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.174+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f81f4051090 con 0x7f81fc0b5280 2023-06-23T18:04:45.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81fc04e4c0 con 0x7f81fc0b5280 2023-06-23T18:04:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f81f97fa700 1 -- 172.21.15.22:0/4145183010 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81f40546b0 con 0x7f81fc0b5280 2023-06-23T18:04:45.355 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 msgr2=0x7f81e80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 0x7f81e80687a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81fc08bc90 tx=0x7f81f0008030 comp rx=0 tx=0).stop 2023-06-23T18:04:45.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 msgr2=0x7f81fc06c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f81f400cb90 tx=0x7f81f400cea0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 shutdown_connections 2023-06-23T18:04:45.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc0b5280 0x7f81fc06c500 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81fc0d1f30 0x7f81fc09fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f81e80662f0 0x7f81e80687a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 --2- 172.21.15.22:0/4145183010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81fc0d0730 0x7f81fc1f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 >> 172.21.15.22:0/4145183010 conn(0x7f81fc06a280 msgr2=0x7f81fc0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 shutdown_connections 2023-06-23T18:04:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.342+0000 7f8201fd5700 1 -- 172.21.15.22:0/4145183010 wait complete. 2023-06-23T18:04:45.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:45.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.474+0000 7f3ade93d700 1 Processor -- start 2023-06-23T18:04:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.474+0000 7f3ade93d700 1 -- start start 2023-06-23T18:04:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ad80707b0 0x7f3ad80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8070c10 0x7f3ad80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80cbaa0 con 0x7f3ad8069960 2023-06-23T18:04:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad8071110 con 0x7f3ad80707b0 2023-06-23T18:04:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad815c310 con 0x7f3ad8070c10 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add93b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3ad8069960 0x7f3ad8069d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51772/0 (socket says 172.21.15.22:51772) 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ad80707b0 0x7f3ad80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add93b700 1 -- 172.21.15.22:0/3038809672 learned_addr learned my addr 172.21.15.22:0/3038809672 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade13c700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8070c10 0x7f3ad80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 -- 172.21.15.22:0/3038809672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8070c10 msgr2=0x7f3ad80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8070c10 0x7f3ad80c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 -- 172.21.15.22:0/3038809672 >> v1:172.21.15.22:6789/0 conn(0x7f3ad8069960 legacy=0x7f3ad8069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 -- 172.21.15.22:0/3038809672 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80cbd80 con 0x7f3ad80707b0 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3add13a700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad80707b0 0x7f3ad80c9710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009880 tx=0x7f3ac8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2365e3bf5f7e6c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3adc939700 1 -- 172.21.15.22:0/3038809672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8011830 con 0x7f3ad80707b0 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3adc939700 1 -- 172.21.15.22:0/3038809672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac8011990 con 0x7f3ad80707b0 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3adc939700 1 -- 172.21.15.22:0/3038809672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8011d20 con 0x7f3ad80707b0 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- 172.21.15.22:0/3038809672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad80707b0 msgr2=0x7f3ad80c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad80707b0 0x7f3ad80c9710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009880 tx=0x7f3ac8009b90 comp rx=0 tx=0).stop 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- 172.21.15.22:0/3038809672 shutdown_connections 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8070c10 0x7f3ad80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3038809672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad80707b0 0x7f3ad80c9710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- 172.21.15.22:0/3038809672 >> 172.21.15.22:0/3038809672 conn(0x7f3ad806b700 msgr2=0x7f3ad80b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- 172.21.15.22:0/3038809672 shutdown_connections 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- 172.21.15.22:0/3038809672 wait complete. 2023-06-23T18:04:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 Processor -- start 2023-06-23T18:04:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.478+0000 7f3ade93d700 1 -- start start 2023-06-23T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8069960 0x7f3ad8153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad8070c10 0x7f3ad815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c8420 con 0x7f3ad815f520 2023-06-23T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81540a0 con 0x7f3ad8070c10 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c7380 con 0x7f3ad8069960 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46782/0 (socket says 172.21.15.22:46782) 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 -- 172.21.15.22:0/3843421488 learned_addr learned my addr 172.21.15.22:0/3843421488 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3add13a700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad8070c10 0x7f3ad815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3add93b700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8069960 0x7f3ad8153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 -- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8069960 msgr2=0x7f3ad8153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8069960 0x7f3ad8153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 -- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad8070c10 msgr2=0x7f3ad815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad8070c10 0x7f3ad815efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 -- 172.21.15.22:0/3843421488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80094b0 con 0x7f3ad815f520 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade13c700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007e60 tx=0x7f3ad000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000ed30 con 0x7f3ad815f520 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad000ee90 con 0x7f3ad815f520 2023-06-23T18:04:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad80c7660 con 0x7f3ad815f520 2023-06-23T18:04:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0014660 con 0x7f3ad815f520 2023-06-23T18:04:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad80c7c20 con 0x7f3ad815f520 2023-06-23T18:04:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.482+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad804e4c0 con 0x7f3ad815f520 2023-06-23T18:04:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.486+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3ad00147c0 con 0x7f3ad815f520 2023-06-23T18:04:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.486+0000 7f3ace7fc700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 0x7f3abc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.486+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3ad0084620 con 0x7f3ad815f520 2023-06-23T18:04:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.486+0000 7f3add93b700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 0x7f3abc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.486+0000 7f3add93b700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 0x7f3abc0687d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000f60 tx=0x7f3ad4007f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.494+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3ad0055290 con 0x7f3ad815f520 2023-06-23T18:04:45.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.658+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ad80cc030 con 0x7f3ad815f520 2023-06-23T18:04:45.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.658+0000 7f3ace7fc700 1 -- 172.21.15.22:0/3843421488 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ad0054e20 con 0x7f3ad815f520 2023-06-23T18:04:45.670 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:45.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 msgr2=0x7f3abc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 0x7f3abc0687d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000f60 tx=0x7f3ad4007f30 comp rx=0 tx=0).stop 2023-06-23T18:04:45.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 msgr2=0x7f3ad815a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007e60 tx=0x7f3ad000e3c0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 shutdown_connections 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad815f520 0x7f3ad815a2d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3abc066320 0x7f3abc0687d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad8070c10 0x7f3ad815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 --2- 172.21.15.22:0/3843421488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ad8069960 0x7f3ad8153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 >> 172.21.15.22:0/3843421488 conn(0x7f3ad806b700 msgr2=0x7f3ad8160070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 shutdown_connections 2023-06-23T18:04:45.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.662+0000 7f3ade93d700 1 -- 172.21.15.22:0/3843421488 wait complete. 2023-06-23T18:04:45.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.794+0000 7f74c759e700 1 Processor -- start 2023-06-23T18:04:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.794+0000 7f74c759e700 1 -- start start 2023-06-23T18:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.794+0000 7f74c759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f74c80b7af0 0x7f74c80ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.794+0000 7f74c759e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80b7ec0 con 0x7f74c80cc9e0 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80b1ad0 con 0x7f74c80cf480 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80d0a10 con 0x7f74c80b7af0 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c5d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f74c80cc9e0 0x7f74c80ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51782/0 (socket says 172.21.15.22:51782) 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c5d9b700 1 -- 172.21.15.22:0/2246136822 learned_addr learned my addr 172.21.15.22:0/2246136822 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c659c700 1 --2- 172.21.15.22:0/2246136822 >> v2:172.21.15.169:3300/0 conn(0x7f74c80b7af0 0x7f74c80ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707772197 0 0) 0x7f74c80b1ad0 con 0x7f74c80cf480 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b40035d0 con 0x7f74c80cf480 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643971284 0 0) 0x7f74c80b7ec0 con 0x7f74c80cc9e0 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c80b1ad0 con 0x7f74c80cc9e0 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c659c700 1 -- 172.21.15.22:0/2246136822 >> v1:172.21.15.169:6789/0 conn(0x7f74c80cf480 legacy=0x7f74c80d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c659c700 1 -- 172.21.15.22:0/2246136822 >> v1:172.21.15.22:6789/0 conn(0x7f74c80cc9e0 legacy=0x7f74c80ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c659c700 1 -- 172.21.15.22:0/2246136822 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80d0cb0 con 0x7f74c80b7af0 2023-06-23T18:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c659c700 1 --2- 172.21.15.22:0/2246136822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80b7af0 0x7f74c80ce540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009d90 tx=0x7f74c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3bf837cd5cc5a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000e070 con 0x7f74c80b7af0 2023-06-23T18:04:45.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000bb20 con 0x7f74c80b7af0 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c559a700 1 -- 172.21.15.22:0/2246136822 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000be90 con 0x7f74c80b7af0 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- 172.21.15.22:0/2246136822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80b7af0 msgr2=0x7f74c80ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 --2- 172.21.15.22:0/2246136822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80b7af0 0x7f74c80ce540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009d90 tx=0x7f74c000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- 172.21.15.22:0/2246136822 shutdown_connections 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 --2- 172.21.15.22:0/2246136822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80b7af0 0x7f74c80ce540 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- 172.21.15.22:0/2246136822 >> 172.21.15.22:0/2246136822 conn(0x7f74c8069960 msgr2=0x7f74c8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- 172.21.15.22:0/2246136822 shutdown_connections 2023-06-23T18:04:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- 172.21.15.22:0/2246136822 wait complete. 2023-06-23T18:04:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 Processor -- start 2023-06-23T18:04:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.798+0000 7f74c759e700 1 -- start start 2023-06-23T18:04:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74c80b7af0 0x7f74c80c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80cc9e0 0x7f74c80c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c8820 con 0x7f74c80cf480 2023-06-23T18:04:45.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81f1600 con 0x7f74c80cc9e0 2023-06-23T18:04:45.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c7240 con 0x7f74c80b7af0 2023-06-23T18:04:45.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46788/0 (socket says 172.21.15.22:46788) 2023-06-23T18:04:45.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 -- 172.21.15.22:0/3094856382 learned_addr learned my addr 172.21.15.22:0/3094856382 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c659c700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74c80b7af0 0x7f74c80c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c5d9b700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80cc9e0 0x7f74c80c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 -- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74c80b7af0 msgr2=0x7f74c80c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74c80b7af0 0x7f74c80c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 -- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80cc9e0 msgr2=0x7f74c80c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80cc9e0 0x7f74c80c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 -- 172.21.15.22:0/3094856382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009a70 con 0x7f74c80cf480 2023-06-23T18:04:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c6d9d700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c3e0 tx=0x7f74b800c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80052d0 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8005430 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80055a0 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80c7520 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80c7ae0 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c804e4c0 con 0x7f74c80cf480 2023-06-23T18:04:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f74b80038e0 con 0x7f74c80cf480 2023-06-23T18:04:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.802+0000 7f74beffd700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 0x7f74a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.806+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f74b8083330 con 0x7f74c80cf480 2023-06-23T18:04:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.806+0000 7f74c659c700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 0x7f74a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.806+0000 7f74c659c700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 0x7f74a80687a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74c0014040 tx=0x7f74c0011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.810+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f74b8050550 con 0x7f74c80cf480 2023-06-23T18:04:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.990+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74c80d0f60 con 0x7f74c80cf480 2023-06-23T18:04:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.990+0000 7f74beffd700 1 -- 172.21.15.22:0/3094856382 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74b8053b70 con 0x7f74c80cf480 2023-06-23T18:04:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 msgr2=0x7f74a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 0x7f74a80687a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74c0014040 tx=0x7f74c0011000 comp rx=0 tx=0).stop 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 msgr2=0x7f74c80c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c3e0 tx=0x7f74b800c6f0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 shutdown_connections 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74c80cf480 0x7f74c80c99a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f74a80662f0 0x7f74a80687a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74c80cc9e0 0x7f74c80c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 --2- 172.21.15.22:0/3094856382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74c80b7af0 0x7f74c80c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 >> 172.21.15.22:0/3094856382 conn(0x7f74c8069960 msgr2=0x7f74c80d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 shutdown_connections 2023-06-23T18:04:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:45.994+0000 7f74c759e700 1 -- 172.21.15.22:0/3094856382 wait complete. 2023-06-23T18:04:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:46.018 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:04:46.018 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:04:46.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:46.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 Processor -- start 2023-06-23T18:04:46.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 -- start start 2023-06-23T18:04:46.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc660160420 0x7fc6600cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660070040 0x7fc6600cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc660163ec0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600cc6f0 con 0x7fc660160420 2023-06-23T18:04:46.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66720c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6601641a0 con 0x7fc660070040 2023-06-23T18:04:46.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc660160420 0x7fc6600cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc66620a700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc6600c0fe0 0x7fc6600c13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51788/0 (socket says 172.21.15.22:51788) 2023-06-23T18:04:46.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc660160420 0x7fc6600cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46798/0 (socket says 172.21.15.22:46798) 2023-06-23T18:04:46.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 -- 172.21.15.22:0/1346951137 learned_addr learned my addr 172.21.15.22:0/1346951137 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:46.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc666a0b700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660070040 0x7fc6600cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 -- 172.21.15.22:0/1346951137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660070040 msgr2=0x7fc6600cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660070040 0x7fc6600cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 -- 172.21.15.22:0/1346951137 >> v1:172.21.15.22:6789/0 conn(0x7fc6600c0fe0 legacy=0x7fc6600c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665208700 1 -- 172.21.15.22:0/1346951137 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146105726 0 0) 0x7fc660163ec0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 -- 172.21.15.22:0/1346951137 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660164480 con 0x7fc660160420 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665a09700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660160420 0x7fc6600cbc70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc650009e50 tx=0x7fc65000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661e9fb03092e9de server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665208700 1 -- 172.21.15.22:0/1346951137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000e070 con 0x7fc660160420 2023-06-23T18:04:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.134+0000 7fc665208700 1 -- 172.21.15.22:0/1346951137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000b9c0 con 0x7fc660160420 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc665208700 1 -- 172.21.15.22:0/1346951137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000bd70 con 0x7fc660160420 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- 172.21.15.22:0/1346951137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660160420 msgr2=0x7fc6600cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660160420 0x7fc6600cbc70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc650009e50 tx=0x7fc65000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- 172.21.15.22:0/1346951137 shutdown_connections 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660070040 0x7fc6600cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 --2- 172.21.15.22:0/1346951137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc660160420 0x7fc6600cbc70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- 172.21.15.22:0/1346951137 >> 172.21.15.22:0/1346951137 conn(0x7fc66006a5f0 msgr2=0x7fc6600caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- 172.21.15.22:0/1346951137 shutdown_connections 2023-06-23T18:04:46.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- 172.21.15.22:0/1346951137 wait complete. 2023-06-23T18:04:46.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 Processor -- start 2023-06-23T18:04:46.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.138+0000 7fc66720c700 1 -- start start 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc660070040 0x7fc6601f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660160420 0x7fc6600ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600c56c0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600b0760 con 0x7fc660070040 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600c6990 con 0x7fc660160420 2023-06-23T18:04:46.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46802/0 (socket says 172.21.15.22:46802) 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 -- 172.21.15.22:0/687665952 learned_addr learned my addr 172.21.15.22:0/687665952 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66620a700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc660070040 0x7fc6601f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc666a0b700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660160420 0x7fc6600ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 -- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660160420 msgr2=0x7fc6600ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660160420 0x7fc6600ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 -- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc660070040 msgr2=0x7fc6601f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc660070040 0x7fc6601f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 -- 172.21.15.22:0/687665952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc650009b30 con 0x7fc6600c0fe0 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc665a09700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc650012000 tx=0x7fc650004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650004900 con 0x7fc6600c0fe0 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6500099b0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6601f59c0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65001b650 con 0x7fc6600c0fe0 2023-06-23T18:04:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.142+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6601f5f80 con 0x7fc6600c0fe0 2023-06-23T18:04:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.146+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6601f6110 con 0x7fc6600c0fe0 2023-06-23T18:04:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.146+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc65001b7b0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.146+0000 7fc65effd700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.146+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc650085530 con 0x7fc6600c0fe0 2023-06-23T18:04:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.146+0000 7fc66620a700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.150+0000 7fc66620a700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 0x7fc64c0687a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc654000f60 tx=0x7fc654007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.154+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc650052710 con 0x7fc6600c0fe0 2023-06-23T18:04:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.322+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc66004e4c0 con 0x7fc6600c0fe0 2023-06-23T18:04:46.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.322+0000 7fc65effd700 1 -- 172.21.15.22:0/687665952 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7fc650055d30 con 0x7fc6600c0fe0 2023-06-23T18:04:46.337 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:46.338 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:46.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 msgr2=0x7fc64c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 0x7fc64c0687a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc654000f60 tx=0x7fc654007e00 comp rx=0 tx=0).stop 2023-06-23T18:04:46.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 msgr2=0x7fc6600b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc650012000 tx=0x7fc650004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 shutdown_connections 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc660160420 0x7fc6600ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6600c0fe0 0x7fc6600b0220 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc64c0662f0 0x7fc64c0687a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 --2- 172.21.15.22:0/687665952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc660070040 0x7fc6601f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 >> 172.21.15.22:0/687665952 conn(0x7fc66006a5f0 msgr2=0x7fc660070410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 shutdown_connections 2023-06-23T18:04:46.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.330+0000 7fc66720c700 1 -- 172.21.15.22:0/687665952 wait complete. 2023-06-23T18:04:46.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:46.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 Processor -- start 2023-06-23T18:04:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 -- start start 2023-06-23T18:04:46.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f64e80d1f30 0x7f64e80d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64e80d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e80ccb30 con 0x7f64e80cf3a0 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e80d2880 con 0x7f64e80d1f30 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ef10e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80d2b60 con 0x7f64e80d2360 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ee10c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f64e80cf3a0 0x7f64e80cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42618/0 (socket says 172.21.15.22:42618) 2023-06-23T18:04:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ed90b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f64e80d1f30 0x7f64e80d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ee10c700 1 -- 172.21.15.22:0/1065266415 learned_addr learned my addr 172.21.15.22:0/1065266415 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.470+0000 7f64ee90d700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64e80d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322269273 0 0) 0x7f64e80ccb30 con 0x7f64e80cf3a0 2023-06-23T18:04:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d40035d0 con 0x7f64e80cf3a0 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3831264726 0 0) 0x7f64d40035d0 con 0x7f64e80cf3a0 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 msgr2=0x7f64e80d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64e80d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 >> v2:172.21.15.22:3300/0 conn(0x7f64e80d1f30 msgr2=0x7f64e80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 --2- 172.21.15.22:0/1065266415 >> v2:172.21.15.22:3300/0 conn(0x7f64e80d1f30 0x7f64e80d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80d2e40 con 0x7f64e80cf3a0 2023-06-23T18:04:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f64e40017f0 con 0x7f64e80cf3a0 2023-06-23T18:04:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 >> v1:172.21.15.22:6789/0 conn(0x7f64e80cf3a0 legacy=0x7f64e80cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4004450 0x7f64d4004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64d4004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80d2880 con 0x7f64d4003a20 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80d2b60 con 0x7f64d4004450 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d400ba30 con 0x7f64e80d2360 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed90b700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4004450 0x7f64d4004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee90d700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64d4004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 -- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 msgr2=0x7f64d4004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:46.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e80d2360 0x7f64d4004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 -- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4004450 msgr2=0x7f64d4004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4004450 0x7f64d4004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 -- 172.21.15.22:0/1065266415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40035d0 con 0x7f64d4003a20 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed90b700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4004450 0x7f64d4004910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee10c700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64d4003e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c270 tx=0x7f64d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ad3815ddc73849 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400ce10 con 0x7f64d4003a20 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4007c10 con 0x7f64d4003a20 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ed10a700 1 -- 172.21.15.22:0/1065266415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4007310 con 0x7f64d4003a20 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 -- 172.21.15.22:0/1065266415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 msgr2=0x7f64d4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 --2- 172.21.15.22:0/1065266415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64d4003e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c270 tx=0x7f64d40058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ee90d700 1 -- 172.21.15.22:0/1065266415 reap_dead start 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 -- 172.21.15.22:0/1065266415 shutdown_connections 2023-06-23T18:04:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 -- 172.21.15.22:0/1065266415 >> 172.21.15.22:0/1065266415 conn(0x7f64e806a280 msgr2=0x7f64e809b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 -- 172.21.15.22:0/1065266415 shutdown_connections 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 -- 172.21.15.22:0/1065266415 wait complete. 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.474+0000 7f64ef10e700 1 Processor -- start 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- start start 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e815c0e0 0x7f64e814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e814e6c0 0x7f64e814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e815f2c0 con 0x7f64d4003a20 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80c87c0 con 0x7f64e814e6c0 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80b6900 con 0x7f64e815c0e0 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41844/0 (socket says 172.21.15.22:41844) 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 -- 172.21.15.22:0/420887570 learned_addr learned my addr 172.21.15.22:0/420887570 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:46.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee90d700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e814e6c0 0x7f64e814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ed90b700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e815c0e0 0x7f64e814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 -- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e815c0e0 msgr2=0x7f64e814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e815c0e0 0x7f64e814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 -- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e814e6c0 msgr2=0x7f64e814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e814e6c0 0x7f64e814eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 -- 172.21.15.22:0/420887570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80b62f0 con 0x7f64d4003a20 2023-06-23T18:04:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ee10c700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c860 tx=0x7f64e4001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4005ce0 con 0x7f64d4003a20 2023-06-23T18:04:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e815efe0 con 0x7f64d4003a20 2023-06-23T18:04:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4005e40 con 0x7f64d4003a20 2023-06-23T18:04:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4007310 con 0x7f64d4003a20 2023-06-23T18:04:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8150980 con 0x7f64d4003a20 2023-06-23T18:04:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.478+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e80b6150 con 0x7f64d4003a20 2023-06-23T18:04:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.482+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f64e4019030 con 0x7f64d4003a20 2023-06-23T18:04:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.482+0000 7f64deffd700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 0x7f64cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.482+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f64e4084fe0 con 0x7f64d4003a20 2023-06-23T18:04:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.482+0000 7f64ed90b700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 0x7f64cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.482+0000 7f64ed90b700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 0x7f64cc0687a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64e80b74d0 tx=0x7f64d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.490+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f64e80b6150 con 0x7f64d4003a20 2023-06-23T18:04:46.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.650+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64e80b6150 con 0x7f64d4003a20 2023-06-23T18:04:46.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.650+0000 7f64deffd700 1 -- 172.21.15.22:0/420887570 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64e80b6150 con 0x7f64d4003a20 2023-06-23T18:04:46.663 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:46.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 msgr2=0x7f64cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 0x7f64cc0687a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64e80b74d0 tx=0x7f64d8008040 comp rx=0 tx=0).stop 2023-06-23T18:04:46.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 msgr2=0x7f64e80b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c860 tx=0x7f64e4001b80 comp rx=0 tx=0).stop 2023-06-23T18:04:46.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 shutdown_connections 2023-06-23T18:04:46.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f64cc0662f0 0x7f64cc0687a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e814e6c0 0x7f64e814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64e815c0e0 0x7f64e814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 --2- 172.21.15.22:0/420887570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64d4003a20 0x7f64e80b6d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 >> 172.21.15.22:0/420887570 conn(0x7f64e806a280 msgr2=0x7f64e80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 shutdown_connections 2023-06-23T18:04:46.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.654+0000 7f64ef10e700 1 -- 172.21.15.22:0/420887570 wait complete. 2023-06-23T18:04:46.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:46.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.782+0000 7fe206d2c700 1 Processor -- start 2023-06-23T18:04:46.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.782+0000 7fe206d2c700 1 -- start start 2023-06-23T18:04:46.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.782+0000 7fe206d2c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 0x7fe2000cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000ccb30 con 0x7fe2000d0730 2023-06-23T18:04:46.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000cfcb0 con 0x7fe2000d1f30 2023-06-23T18:04:46.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000d2880 con 0x7fe2000cf3a0 2023-06-23T18:04:46.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe20652b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe2000d0730 0x7fe2000ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42634/0 (socket says 172.21.15.22:42634) 2023-06-23T18:04:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe20652b700 1 -- 172.21.15.22:0/4263520644 learned_addr learned my addr 172.21.15.22:0/4263520644 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe205d2a700 1 --2- 172.21.15.22:0/4263520644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 0x7fe2000cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690561566 0 0) 0x7fe2000ccb30 con 0x7fe2000d0730 2023-06-23T18:04:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ec0035d0 con 0x7fe2000d0730 2023-06-23T18:04:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133760698 0 0) 0x7fe2000cfcb0 con 0x7fe2000d1f30 2023-06-23T18:04:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2000ccb30 con 0x7fe2000d1f30 2023-06-23T18:04:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406737309 0 0) 0x7fe1ec0035d0 con 0x7fe2000d0730 2023-06-23T18:04:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe205d2a700 1 -- 172.21.15.22:0/4263520644 >> v1:172.21.15.169:6789/0 conn(0x7fe2000d1f30 legacy=0x7fe2000d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe205d2a700 1 -- 172.21.15.22:0/4263520644 >> v1:172.21.15.22:6789/0 conn(0x7fe2000d0730 legacy=0x7fe2000ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe205d2a700 1 -- 172.21.15.22:0/4263520644 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000d2b60 con 0x7fe2000cf3a0 2023-06-23T18:04:46.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe205d2a700 1 --2- 172.21.15.22:0/4263520644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 0x7fe2000cf770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009870 tx=0x7fe1f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2019a86c68d8a66 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80019f0 con 0x7fe2000cf3a0 2023-06-23T18:04:46.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0004030 con 0x7fe2000cf3a0 2023-06-23T18:04:46.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe204d28700 1 -- 172.21.15.22:0/4263520644 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00043c0 con 0x7fe2000cf3a0 2023-06-23T18:04:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- 172.21.15.22:0/4263520644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 msgr2=0x7fe2000cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 --2- 172.21.15.22:0/4263520644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 0x7fe2000cf770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009870 tx=0x7fe1f0009b80 comp rx=0 tx=0).stop 2023-06-23T18:04:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- 172.21.15.22:0/4263520644 shutdown_connections 2023-06-23T18:04:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 --2- 172.21.15.22:0/4263520644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000cf3a0 0x7fe2000cf770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- 172.21.15.22:0/4263520644 >> 172.21.15.22:0/4263520644 conn(0x7fe20006a280 msgr2=0x7fe20009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- 172.21.15.22:0/4263520644 shutdown_connections 2023-06-23T18:04:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.786+0000 7fe206d2c700 1 -- 172.21.15.22:0/4263520644 wait complete. 2023-06-23T18:04:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 Processor -- start 2023-06-23T18:04:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 -- start start 2023-06-23T18:04:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 0x7fe200155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000d1f30 0x7fe2000b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c8760 con 0x7fe2000cf3a0 2023-06-23T18:04:46.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200155d40 con 0x7fe2000d0730 2023-06-23T18:04:46.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe206d2c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000b80b0 con 0x7fe2000d1f30 2023-06-23T18:04:46.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41852/0 (socket says 172.21.15.22:41852) 2023-06-23T18:04:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 -- 172.21.15.22:0/1204127853 learned_addr learned my addr 172.21.15.22:0/1204127853 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205529700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 0x7fe200155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe20652b700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000d1f30 0x7fe2000b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 -- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000d1f30 msgr2=0x7fe2000b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000d1f30 0x7fe2000b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 -- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 msgr2=0x7fe200155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 0x7fe200155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205d2a700 1 -- 172.21.15.22:0/1204127853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009450 con 0x7fe2000cf3a0 2023-06-23T18:04:46.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.790+0000 7fe205529700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 0x7fe200155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:46.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe205d2a700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009ed0 tx=0x7fe1f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0003c00 con 0x7fe2000cf3a0 2023-06-23T18:04:46.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0020070 con 0x7fe2000cf3a0 2023-06-23T18:04:46.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0011630 con 0x7fe2000cf3a0 2023-06-23T18:04:46.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000b6ad0 con 0x7fe2000cf3a0 2023-06-23T18:04:46.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000b7090 con 0x7fe2000cf3a0 2023-06-23T18:04:46.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20004e4c0 con 0x7fe2000cf3a0 2023-06-23T18:04:46.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe1f0022030 con 0x7fe2000cf3a0 2023-06-23T18:04:46.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 0x7fe1e40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:46.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe1f0085170 con 0x7fe2000cf3a0 2023-06-23T18:04:46.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.794+0000 7fe205529700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 0x7fe1e40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:46.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.798+0000 7fe205529700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 0x7fe1e40687d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2000cf030 tx=0x7fe1fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:46.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.802+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe1f0052350 con 0x7fe2000cf3a0 2023-06-23T18:04:46.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.966+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe20006b580 con 0x7fe2000cf3a0 2023-06-23T18:04:46.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.966+0000 7fe1f67fc700 1 -- 172.21.15.22:0/1204127853 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1f0055970 con 0x7fe2000cf3a0 2023-06-23T18:04:46.980 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:46.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 msgr2=0x7fe1e40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 0x7fe1e40687d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2000cf030 tx=0x7fe1fc009040 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 msgr2=0x7fe20015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009ed0 tx=0x7fe1f0004770 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 shutdown_connections 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2000d1f30 0x7fe2000b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe1e4066320 0x7fe1e40687d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2000d0730 0x7fe200155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 --2- 172.21.15.22:0/1204127853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000cf3a0 0x7fe20015c0e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:46.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 >> 172.21.15.22:0/1204127853 conn(0x7fe20006a280 msgr2=0x7fe20006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:46.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 shutdown_connections 2023-06-23T18:04:46.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:46.970+0000 7fe206d2c700 1 -- 172.21.15.22:0/1204127853 wait complete. 2023-06-23T18:04:46.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:47.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.102+0000 7fcacce29700 1 Processor -- start 2023-06-23T18:04:47.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.102+0000 7fcacce29700 1 -- start start 2023-06-23T18:04:47.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcac8158b20 0x7fcac806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 0x7fcac80b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac806eab0 con 0x7fcac8070740 2023-06-23T18:04:47.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac80cdf20 con 0x7fcac8158b20 2023-06-23T18:04:47.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80cd680 con 0x7fcac8158ef0 2023-06-23T18:04:47.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac7fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcac8070740 0x7fcac815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42640/0 (socket says 172.21.15.22:42640) 2023-06-23T18:04:47.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac7fff700 1 -- 172.21.15.22:0/3904372174 learned_addr learned my addr 172.21.15.22:0/3904372174 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac77fe700 1 --2- 172.21.15.22:0/3904372174 >> v2:172.21.15.169:3300/0 conn(0x7fcac8158b20 0x7fcac806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 --2- 172.21.15.22:0/3904372174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 0x7fcac80b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac6ffd700 1 -- 172.21.15.22:0/3904372174 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776423920 0 0) 0x7fcac806eab0 con 0x7fcac8070740 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac6ffd700 1 -- 172.21.15.22:0/3904372174 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80035d0 con 0x7fcac8070740 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 -- 172.21.15.22:0/3904372174 >> v2:172.21.15.169:3300/0 conn(0x7fcac8158b20 msgr2=0x7fcac806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:47.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 --2- 172.21.15.22:0/3904372174 >> v2:172.21.15.169:3300/0 conn(0x7fcac8158b20 0x7fcac806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 -- 172.21.15.22:0/3904372174 >> v1:172.21.15.22:6789/0 conn(0x7fcac8070740 legacy=0x7fcac815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 -- 172.21.15.22:0/3904372174 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80cd960 con 0x7fcac8158ef0 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcabffff700 1 --2- 172.21.15.22:0/3904372174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 0x7fcac80b1780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009850 tx=0x7fcab0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb24a64e59b611f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac6ffd700 1 -- 172.21.15.22:0/3904372174 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011830 con 0x7fcac8158ef0 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac6ffd700 1 -- 172.21.15.22:0/3904372174 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0011990 con 0x7fcac8158ef0 2023-06-23T18:04:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcac6ffd700 1 -- 172.21.15.22:0/3904372174 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011d00 con 0x7fcac8158ef0 2023-06-23T18:04:47.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- 172.21.15.22:0/3904372174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 msgr2=0x7fcac80b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 --2- 172.21.15.22:0/3904372174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 0x7fcac80b1780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009850 tx=0x7fcab0009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:47.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- 172.21.15.22:0/3904372174 shutdown_connections 2023-06-23T18:04:47.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 --2- 172.21.15.22:0/3904372174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8158ef0 0x7fcac80b1780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 --2- 172.21.15.22:0/3904372174 >> v2:172.21.15.169:3300/0 conn(0x7fcac8158b20 0x7fcac806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.106+0000 7fcacce29700 1 -- 172.21.15.22:0/3904372174 >> 172.21.15.22:0/3904372174 conn(0x7fcac806b700 msgr2=0x7fcac80b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- 172.21.15.22:0/3904372174 shutdown_connections 2023-06-23T18:04:47.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- 172.21.15.22:0/3904372174 wait complete. 2023-06-23T18:04:47.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 Processor -- start 2023-06-23T18:04:47.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- start start 2023-06-23T18:04:47.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8070740 0x7fcac80c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac8158ef0 0x7fcac80cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c4c40 con 0x7fcac8158b20 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac8153100 con 0x7fcac8158ef0 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcacce29700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80b4480 con 0x7fcac8070740 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcabffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcabffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41866/0 (socket says 172.21.15.22:41866) 2023-06-23T18:04:47.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.110+0000 7fcabffff700 1 -- 172.21.15.22:0/2513656460 learned_addr learned my addr 172.21.15.22:0/2513656460 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:47.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac7fff700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac8158ef0 0x7fcac80cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac77fe700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8070740 0x7fcac80c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 -- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8070740 msgr2=0x7fcac80c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8070740 0x7fcac80c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 -- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac8158ef0 msgr2=0x7fcac80cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac8158ef0 0x7fcac80cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 -- 172.21.15.22:0/2513656460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094b0 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcabffff700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcab00090b0 tx=0x7fcab0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004550 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab001e070 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac80b4f20 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003900 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac80b4720 con 0x7fcac8158b20 2023-06-23T18:04:47.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac80b50b0 con 0x7fcac8158b20 2023-06-23T18:04:47.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.114+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcab0003a60 con 0x7fcac8158b20 2023-06-23T18:04:47.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.118+0000 7fcac4ff9700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 0x7fcaac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.118+0000 7fcac77fe700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 0x7fcaac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.118+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcab0084ff0 con 0x7fcac8158b20 2023-06-23T18:04:47.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.118+0000 7fcac77fe700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 0x7fcaac0687a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcac8153600 tx=0x7fcab8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.122+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fcab00b5700 con 0x7fcac8158b20 2023-06-23T18:04:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcac804e4c0 con 0x7fcac8158b20 2023-06-23T18:04:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcac4ff9700 1 -- 172.21.15.22:0/2513656460 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcab00520d0 con 0x7fcac8158b20 2023-06-23T18:04:47.302 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:47.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 msgr2=0x7fcaac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 0x7fcaac0687a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcac8153600 tx=0x7fcab8008040 comp rx=0 tx=0).stop 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 msgr2=0x7fcac8152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcab00090b0 tx=0x7fcab0004770 comp rx=0 tx=0).stop 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 shutdown_connections 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcaac0662f0 0x7fcaac0687a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac8158ef0 0x7fcac80cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcac8158b20 0x7fcac8152bc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 --2- 172.21.15.22:0/2513656460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcac8070740 0x7fcac80c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 >> 172.21.15.22:0/2513656460 conn(0x7fcac806b700 msgr2=0x7fcac80c4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 shutdown_connections 2023-06-23T18:04:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.290+0000 7fcacce29700 1 -- 172.21.15.22:0/2513656460 wait complete. 2023-06-23T18:04:47.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:47.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 Processor -- start 2023-06-23T18:04:47.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 -- start start 2023-06-23T18:04:47.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340cf3a0 0x7fab340cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fab340d0730 0x7fab340ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab340ccb30 con 0x7fab340d1f30 2023-06-23T18:04:47.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340cfcb0 con 0x7fab340d0730 2023-06-23T18:04:47.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340d2880 con 0x7fab340cf3a0 2023-06-23T18:04:47.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fab340d0730 0x7fab340ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab31d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fab340d1f30 0x7fab340d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42646/0 (socket says 172.21.15.22:42646) 2023-06-23T18:04:47.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab31d9b700 1 -- 172.21.15.22:0/821280117 learned_addr learned my addr 172.21.15.22:0/821280117 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:47.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3259c700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340cf3a0 0x7fab340cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3159a700 1 -- 172.21.15.22:0/821280117 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210084556 0 0) 0x7fab340ccb30 con 0x7fab340d1f30 2023-06-23T18:04:47.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab3159a700 1 -- 172.21.15.22:0/821280117 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab200035d0 con 0x7fab340d1f30 2023-06-23T18:04:47.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 -- 172.21.15.22:0/821280117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340cf3a0 msgr2=0x7fab340cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:47.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340cf3a0 0x7fab340cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 -- 172.21.15.22:0/821280117 >> v1:172.21.15.22:6789/0 conn(0x7fab340d1f30 legacy=0x7fab340d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 -- 172.21.15.22:0/821280117 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340d2b60 con 0x7fab340d0730 2023-06-23T18:04:47.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.430+0000 7fab32d9d700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d0730 0x7fab340ccfc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d90 tx=0x7fab2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d145406f096d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3159a700 1 -- 172.21.15.22:0/821280117 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00e070 con 0x7fab340d0730 2023-06-23T18:04:47.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3159a700 1 -- 172.21.15.22:0/821280117 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c00bb20 con 0x7fab340d0730 2023-06-23T18:04:47.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3159a700 1 -- 172.21.15.22:0/821280117 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c012430 con 0x7fab340d0730 2023-06-23T18:04:47.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- 172.21.15.22:0/821280117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d0730 msgr2=0x7fab340ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d0730 0x7fab340ccfc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d90 tx=0x7fab2c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:47.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- 172.21.15.22:0/821280117 shutdown_connections 2023-06-23T18:04:47.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d0730 0x7fab340ccfc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 --2- 172.21.15.22:0/821280117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340cf3a0 0x7fab340cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- 172.21.15.22:0/821280117 >> 172.21.15.22:0/821280117 conn(0x7fab3406a280 msgr2=0x7fab3409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- 172.21.15.22:0/821280117 shutdown_connections 2023-06-23T18:04:47.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- 172.21.15.22:0/821280117 wait complete. 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 Processor -- start 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.434+0000 7fab3359e700 1 -- start start 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab340cf3a0 0x7fab340c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340d0730 0x7fab340b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b7de0 con 0x7fab340d1f30 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b7160 con 0x7fab340cf3a0 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b7440 con 0x7fab340d0730 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41880/0 (socket says 172.21.15.22:41880) 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 -- 172.21.15.22:0/1803209974 learned_addr learned my addr 172.21.15.22:0/1803209974 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab31d9b700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340d0730 0x7fab340b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3259c700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab340cf3a0 0x7fab340c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 -- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340d0730 msgr2=0x7fab340b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340d0730 0x7fab340b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 -- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab340cf3a0 msgr2=0x7fab340c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab340cf3a0 0x7fab340c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 -- 172.21.15.22:0/1803209974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009a70 con 0x7fab340d1f30 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab32d9d700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab2c014040 tx=0x7fab2c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c012430 con 0x7fab340d1f30 2023-06-23T18:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c020070 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340b7720 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0098f0 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab340c8690 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.438+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab340b7f20 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.442+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fab2c004040 con 0x7fab340d1f30 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.442+0000 7fab2affd700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 0x7fab140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.442+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fab2c022070 con 0x7fab340d1f30 2023-06-23T18:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.442+0000 7fab3259c700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 0x7fab140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.442+0000 7fab3259c700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 0x7fab140687a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab24009be0 tx=0x7fab240091c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.450+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fab2c056750 con 0x7fab340d1f30 2023-06-23T18:04:47.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.606+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab3404e4c0 con 0x7fab340d1f30 2023-06-23T18:04:47.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.606+0000 7fab2affd700 1 -- 172.21.15.22:0/1803209974 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab2c02c020 con 0x7fab340d1f30 2023-06-23T18:04:47.621 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 msgr2=0x7fab140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 0x7fab140687a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab24009be0 tx=0x7fab240091c0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 msgr2=0x7fab340b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab2c014040 tx=0x7fab2c004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 shutdown_connections 2023-06-23T18:04:47.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab340d1f30 0x7fab340b43f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab340d0730 0x7fab340b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fab140662f0 0x7fab140687a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 --2- 172.21.15.22:0/1803209974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab340cf3a0 0x7fab340c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 >> 172.21.15.22:0/1803209974 conn(0x7fab3406a280 msgr2=0x7fab3406b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 shutdown_connections 2023-06-23T18:04:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.610+0000 7fab3359e700 1 -- 172.21.15.22:0/1803209974 wait complete. 2023-06-23T18:04:47.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 Processor -- start 2023-06-23T18:04:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 -- start start 2023-06-23T18:04:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe14815c2e0 0x7fe1480c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14806c6e0 con 0x7fe14815bf10 2023-06-23T18:04:47.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14806cf70 con 0x7fe14815c2e0 2023-06-23T18:04:47.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14e285700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480cddf0 con 0x7fe14806f660 2023-06-23T18:04:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14ca82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.742+0000 7fe14da84700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe14815c2e0 0x7fe1480c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14ca82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:46188/0 (socket says 172.21.15.22:46188) 2023-06-23T18:04:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14ca82700 1 -- 172.21.15.22:0/2234456104 learned_addr learned my addr 172.21.15.22:0/2234456104 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14da84700 1 -- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 msgr2=0x7fe1480c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14da84700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14da84700 1 -- 172.21.15.22:0/2234456104 >> v1:172.21.15.169:6789/0 conn(0x7fe14815bf10 legacy=0x7fe14806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14da84700 1 -- 172.21.15.22:0/2234456104 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14806c9b0 con 0x7fe14815c2e0 2023-06-23T18:04:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14ca82700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14da84700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480c8f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe140009d90 tx=0x7fe14000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2831e830283e5a67 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe13ffff700 1 -- 172.21.15.22:0/2234456104 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000e070 con 0x7fe14815c2e0 2023-06-23T18:04:47.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe13ffff700 1 -- 172.21.15.22:0/2234456104 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14000bb20 con 0x7fe14815c2e0 2023-06-23T18:04:47.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe13ffff700 1 -- 172.21.15.22:0/2234456104 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140012430 con 0x7fe14815c2e0 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 -- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 msgr2=0x7fe1480c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480c8f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe140009d90 tx=0x7fe14000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 -- 172.21.15.22:0/2234456104 shutdown_connections 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480c8f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 --2- 172.21.15.22:0/2234456104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 -- 172.21.15.22:0/2234456104 >> 172.21.15.22:0/2234456104 conn(0x7fe14806a5f0 msgr2=0x7fe1480caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 -- 172.21.15.22:0/2234456104 shutdown_connections 2023-06-23T18:04:47.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.746+0000 7fe14e285700 1 -- 172.21.15.22:0/2234456104 wait complete. 2023-06-23T18:04:47.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.750+0000 7fe14e285700 1 Processor -- start 2023-06-23T18:04:47.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.750+0000 7fe14e285700 1 -- start start 2023-06-23T18:04:47.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.750+0000 7fe14e285700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.750+0000 7fe14e285700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481f2710 con 0x7fe14815bf10 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481f29b0 con 0x7fe14815c2e0 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481f2c90 con 0x7fe14806f660 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41894/0 (socket says 172.21.15.22:41894) 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 -- 172.21.15.22:0/4042354565 learned_addr learned my addr 172.21.15.22:0/4042354565 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14d283700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14da84700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 -- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 msgr2=0x7fe1480b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 -- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 msgr2=0x7fe1480cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 -- 172.21.15.22:0/4042354565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009570 con 0x7fe14815bf10 2023-06-23T18:04:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14da84700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14ca82700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e9d0 tx=0x7fe14400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400c850 con 0x7fe14815bf10 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400c9b0 con 0x7fe14815bf10 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe140009a70 con 0x7fe14815bf10 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400cd20 con 0x7fe14815bf10 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1481f3250 con 0x7fe14815bf10 2023-06-23T18:04:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.754+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14804e4c0 con 0x7fe14815bf10 2023-06-23T18:04:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.758+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe144013470 con 0x7fe14815bf10 2023-06-23T18:04:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.758+0000 7fe13dffb700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 0x7fe12c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.758+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe144083d10 con 0x7fe14815bf10 2023-06-23T18:04:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.758+0000 7fe14d283700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 0x7fe12c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.758+0000 7fe14d283700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 0x7fe12c0687a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe138000f60 tx=0x7fe138009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:47.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.762+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe144050ef0 con 0x7fe14815bf10 2023-06-23T18:04:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.926+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1480b3ff0 con 0x7fe14815bf10 2023-06-23T18:04:47.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.926+0000 7fe13dffb700 1 -- 172.21.15.22:0/4042354565 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe144054510 con 0x7fe14815bf10 2023-06-23T18:04:47.940 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 msgr2=0x7fe12c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 0x7fe12c0687a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe138000f60 tx=0x7fe138009040 comp rx=0 tx=0).stop 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 msgr2=0x7fe1481500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e9d0 tx=0x7fe14400ece0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 shutdown_connections 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe12c0662f0 0x7fe12c0687a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14815c2e0 0x7fe1480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe14815bf10 0x7fe1481500c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 --2- 172.21.15.22:0/4042354565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe14806f660 0x7fe1480b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:47.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 >> 172.21.15.22:0/4042354565 conn(0x7fe14806a5f0 msgr2=0x7fe1480aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:47.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 shutdown_connections 2023-06-23T18:04:47.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:47.930+0000 7fe14e285700 1 -- 172.21.15.22:0/4042354565 wait complete. 2023-06-23T18:04:47.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:04:48.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.054+0000 7f7ef7fff700 1 Processor -- start 2023-06-23T18:04:48.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.054+0000 7f7ef7fff700 1 -- start start 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ef8070280 0x7f7ef80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ef80c9570 0x7f7ef806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef80c9b50 con 0x7f7ef80affd0 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef81590a0 con 0x7f7ef8070280 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef80b0450 con 0x7f7ef80c9570 2023-06-23T18:04:48.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ef8070280 0x7f7ef80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7ef8070280 0x7f7ef80afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41908/0 (socket says 172.21.15.22:41908) 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 -- 172.21.15.22:0/554311397 learned_addr learned my addr 172.21.15.22:0/554311397 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef77fe700 1 --2- 172.21.15.22:0/554311397 >> v2:172.21.15.169:3300/0 conn(0x7f7ef80c9570 0x7f7ef806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 -- 172.21.15.22:0/554311397 >> v2:172.21.15.169:3300/0 conn(0x7f7ef80c9570 msgr2=0x7f7ef806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 --2- 172.21.15.22:0/554311397 >> v2:172.21.15.169:3300/0 conn(0x7f7ef80c9570 0x7f7ef806b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 -- 172.21.15.22:0/554311397 >> v1:172.21.15.169:6789/0 conn(0x7f7ef80affd0 legacy=0x7f7ef80c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 -- 172.21.15.22:0/554311397 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80c9e30 con 0x7f7ef8070280 2023-06-23T18:04:48.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef6ffd700 1 --2- 172.21.15.22:0/554311397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef8070280 0x7f7ef80afa90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ef806d0b0 tx=0x7f7eec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a61ab883b2d6dc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef5ffb700 1 -- 172.21.15.22:0/554311397 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011830 con 0x7f7ef8070280 2023-06-23T18:04:48.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef5ffb700 1 -- 172.21.15.22:0/554311397 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec011990 con 0x7f7ef8070280 2023-06-23T18:04:48.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef5ffb700 1 -- 172.21.15.22:0/554311397 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011d20 con 0x7f7ef8070280 2023-06-23T18:04:48.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- 172.21.15.22:0/554311397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef8070280 msgr2=0x7f7ef80afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/554311397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef8070280 0x7f7ef80afa90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ef806d0b0 tx=0x7f7eec009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- 172.21.15.22:0/554311397 shutdown_connections 2023-06-23T18:04:48.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/554311397 >> v2:172.21.15.169:3300/0 conn(0x7f7ef80c9570 0x7f7ef806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/554311397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef8070280 0x7f7ef80afa90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- 172.21.15.22:0/554311397 >> 172.21.15.22:0/554311397 conn(0x7f7ef806a5f0 msgr2=0x7f7ef80caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- 172.21.15.22:0/554311397 shutdown_connections 2023-06-23T18:04:48.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.058+0000 7f7ef7fff700 1 -- 172.21.15.22:0/554311397 wait complete. 2023-06-23T18:04:48.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 Processor -- start 2023-06-23T18:04:48.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 -- start start 2023-06-23T18:04:48.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070280 0x7f7ef809ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 0x7f7ef80c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef809b7c0 con 0x7f7ef80c9570 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef81591e0 con 0x7f7ef80affd0 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef7fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef81594c0 con 0x7f7ef8070280 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41914/0 (socket says 172.21.15.22:41914) 2023-06-23T18:04:48.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef6ffd700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070280 0x7f7ef809ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.062+0000 7f7ef77fe700 1 -- 172.21.15.22:0/462467768 learned_addr learned my addr 172.21.15.22:0/462467768 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:48.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef67fc700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 0x7f7ef80c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 -- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070280 msgr2=0x7f7ef809ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070280 0x7f7ef809ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 -- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 msgr2=0x7f7ef80c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 0x7f7ef80c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 -- 172.21.15.22:0/462467768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009570 con 0x7f7ef80c9570 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef67fc700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 0x7f7ef80c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef77fe700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000efc0 tx=0x7f7ef000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000cb80 con 0x7f7ef80c9570 2023-06-23T18:04:48.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef000f040 con 0x7f7ef80c9570 2023-06-23T18:04:48.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0014480 con 0x7f7ef80c9570 2023-06-23T18:04:48.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef81597a0 con 0x7f7ef80c9570 2023-06-23T18:04:48.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef809a5a0 con 0x7f7ef80c9570 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef8159cc0 con 0x7f7ef80c9570 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.066+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7ef00145e0 con 0x7f7ef80c9570 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.070+0000 7f7edf7fe700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 0x7f7ee40687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.070+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7ef0084840 con 0x7f7ef80c9570 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.070+0000 7f7ef6ffd700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 0x7f7ee40687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.070+0000 7f7ef6ffd700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 0x7f7ee40687d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.074+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7ef8159cc0 con 0x7f7ef80c9570 2023-06-23T18:04:48.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.246+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ef8159cc0 con 0x7f7ef80c9570 2023-06-23T18:04:48.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7edf7fe700 1 -- 172.21.15.22:0/462467768 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7ef8159cc0 con 0x7f7ef80c9570 2023-06-23T18:04:48.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 msgr2=0x7f7ee40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 0x7f7ee40687d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec008030 comp rx=0 tx=0).stop 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 msgr2=0x7f7ef80c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000efc0 tx=0x7f7ef000c0c0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 shutdown_connections 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ef80c9570 0x7f7ef80c4290 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7ee4066320 0x7f7ee40687d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef80affd0 0x7f7ef80c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 --2- 172.21.15.22:0/462467768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070280 0x7f7ef809ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 >> 172.21.15.22:0/462467768 conn(0x7f7ef806a5f0 msgr2=0x7f7ef80aada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 shutdown_connections 2023-06-23T18:04:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.250+0000 7f7ef7fff700 1 -- 172.21.15.22:0/462467768 wait complete. 2023-06-23T18:04:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-06-23T18:04:48.276 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi169.front.sepia.ceph.com... 2023-06-23T18:04:48.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:48.277 DEBUG:teuthology.orchestra.run.smithi169:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-06-23T18:04:48.295 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:48.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:04:48.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:04:48.296 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-23T18:04:48.302 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-06-23T18:04:48.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:48.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 Processor -- start 2023-06-23T18:04:48.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 -- start start 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdf680cc380 0x7fdf6816b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf68163ec0 con 0x7fdf68160420 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf681641a0 con 0x7fdf680c0fe0 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.406+0000 7fdf6ecf2700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf68164480 con 0x7fdf680cc380 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6d4ef700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdf68160420 0x7fdf680cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42662/0 (socket says 172.21.15.22:42662) 2023-06-23T18:04:48.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6d4ef700 1 -- 172.21.15.22:0/2069843334 learned_addr learned my addr 172.21.15.22:0/2069843334 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:48.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6e4f1700 1 --2- 172.21.15.22:0/2069843334 >> v2:172.21.15.169:3300/0 conn(0x7fdf680cc380 0x7fdf6816b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142576898 0 0) 0x7fdf68163ec0 con 0x7fdf68160420 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf540035d0 con 0x7fdf68160420 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/2069843334 >> v1:172.21.15.169:6789/0 conn(0x7fdf680c0fe0 legacy=0x7fdf680c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/2069843334 >> v1:172.21.15.22:6789/0 conn(0x7fdf68160420 legacy=0x7fdf680cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084382067 0 0) 0x7fdf540035d0 con 0x7fdf68160420 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/2069843334 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68164760 con 0x7fdf680cc380 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6e4f1700 1 --2- 172.21.15.22:0/2069843334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf6816b050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009d90 tx=0x7fdf6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78924b02b6adfbca server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000e070 con 0x7fdf680cc380 2023-06-23T18:04:48.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60003a40 con 0x7fdf680cc380 2023-06-23T18:04:48.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ccee700 1 -- 172.21.15.22:0/2069843334 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64001690 con 0x7fdf680cc380 2023-06-23T18:04:48.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/2069843334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 msgr2=0x7fdf6816b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/2069843334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf6816b050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009d90 tx=0x7fdf6000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:48.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/2069843334 shutdown_connections 2023-06-23T18:04:48.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/2069843334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf6816b050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/2069843334 >> 172.21.15.22:0/2069843334 conn(0x7fdf6806a5f0 msgr2=0x7fdf680caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/2069843334 shutdown_connections 2023-06-23T18:04:48.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.410+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/2069843334 wait complete. 2023-06-23T18:04:48.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 Processor -- start 2023-06-23T18:04:48.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 -- start start 2023-06-23T18:04:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 0x7fdf680c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf680c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680c7930 con 0x7fdf68160420 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680c8290 con 0x7fdf680cc380 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6ecf2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680c6f90 con 0x7fdf680c0fe0 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41928/0 (socket says 172.21.15.22:41928) 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/1651632100 learned_addr learned my addr 172.21.15.22:0/1651632100 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6d4ef700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf680c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6dcf0700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 0x7fdf680c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 msgr2=0x7fdf680c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 0x7fdf680c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 msgr2=0x7fdf680c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf680c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6e4f1700 1 -- 172.21.15.22:0/1651632100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60009a70 con 0x7fdf68160420 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.414+0000 7fdf6dcf0700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 0x7fdf680c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf6e4f1700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b680 tx=0x7fdf60004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600045c0 con 0x7fdf68160420 2023-06-23T18:04:48.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60004720 con 0x7fdf68160420 2023-06-23T18:04:48.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf680c7270 con 0x7fdf68160420 2023-06-23T18:04:48.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6001d470 con 0x7fdf68160420 2023-06-23T18:04:48.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68099250 con 0x7fdf68160420 2023-06-23T18:04:48.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf680c7a70 con 0x7fdf68160420 2023-06-23T18:04:48.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.418+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fdf6001d5d0 con 0x7fdf68160420 2023-06-23T18:04:48.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.422+0000 7fdf5e7fc700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 0x7fdf4c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.422+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdf60085c80 con 0x7fdf68160420 2023-06-23T18:04:48.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.422+0000 7fdf6dcf0700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 0x7fdf4c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.422+0000 7fdf6dcf0700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 0x7fdf4c0687a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf580017a0 tx=0x7fdf58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.430+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fdf680c7a70 con 0x7fdf68160420 2023-06-23T18:04:48.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.630+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf680c7a70 con 0x7fdf68160420 2023-06-23T18:04:48.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.630+0000 7fdf5e7fc700 1 -- 172.21.15.22:0/1651632100 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fdf680c7a70 con 0x7fdf68160420 2023-06-23T18:04:48.645 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:48.645 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:48.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 msgr2=0x7fdf4c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 0x7fdf4c0687a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf580017a0 tx=0x7fdf58009040 comp rx=0 tx=0).stop 2023-06-23T18:04:48.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 msgr2=0x7fdf68098d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b680 tx=0x7fdf60004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 shutdown_connections 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf68160420 0x7fdf68098d10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fdf4c0662f0 0x7fdf4c0687a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680cc380 0x7fdf680c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 --2- 172.21.15.22:0/1651632100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf680c0fe0 0x7fdf680c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 >> 172.21.15.22:0/1651632100 conn(0x7fdf6806a5f0 msgr2=0x7fdf680ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 shutdown_connections 2023-06-23T18:04:48.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.634+0000 7fdf6ecf2700 1 -- 172.21.15.22:0/1651632100 wait complete. 2023-06-23T18:04:48.649 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:48.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:48.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 Processor -- start 2023-06-23T18:04:48.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 -- start start 2023-06-23T18:04:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 0x7f86c40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86c40c9c60 0x7f86c40ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40cca70 con 0x7f86c4157f00 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40cdb20 con 0x7f86c40cd160 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86caf3c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40c9780 con 0x7f86c40c9c60 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86c9739700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 0x7f86c40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86ca73b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86c40c9c60 0x7f86c40ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86c9739700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 0x7f86c40cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41930/0 (socket says 172.21.15.22:41930) 2023-06-23T18:04:48.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86c9739700 1 -- 172.21.15.22:0/443602118 learned_addr learned my addr 172.21.15.22:0/443602118 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:48.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.762+0000 7f86c9f3a700 1 --1- 172.21.15.22:0/443602118 >> v1:172.21.15.169:6789/0 conn(0x7f86c4157f00 0x7f86c40cc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:52884/0 (socket says 172.21.15.22:52884) 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86ca73b700 1 -- 172.21.15.22:0/443602118 >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 msgr2=0x7f86c40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86ca73b700 1 --2- 172.21.15.22:0/443602118 >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 0x7f86c40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86ca73b700 1 -- 172.21.15.22:0/443602118 >> v1:172.21.15.169:6789/0 conn(0x7f86c4157f00 legacy=0x7f86c40cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86c8f38700 1 -- 172.21.15.22:0/443602118 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810176598 0 0) 0x7f86c40cca70 con 0x7f86c4157f00 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86ca73b700 1 -- 172.21.15.22:0/443602118 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40c4c50 con 0x7f86c40c9c60 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86ca73b700 1 --2- 172.21.15.22:0/443602118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40c9c60 0x7f86c40ca0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b12c0 tx=0x7f86bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4009d6788b98f09e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86c8f38700 1 -- 172.21.15.22:0/443602118 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00e070 con 0x7f86c40c9c60 2023-06-23T18:04:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86c8f38700 1 -- 172.21.15.22:0/443602118 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00bb20 con 0x7f86c40c9c60 2023-06-23T18:04:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86c8f38700 1 -- 172.21.15.22:0/443602118 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc012430 con 0x7f86c40c9c60 2023-06-23T18:04:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 -- 172.21.15.22:0/443602118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40c9c60 msgr2=0x7f86c40ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 --2- 172.21.15.22:0/443602118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40c9c60 0x7f86c40ca0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b12c0 tx=0x7f86bc00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 -- 172.21.15.22:0/443602118 shutdown_connections 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 --2- 172.21.15.22:0/443602118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40c9c60 0x7f86c40ca0e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 --2- 172.21.15.22:0/443602118 >> v2:172.21.15.22:3300/0 conn(0x7f86c40cd160 0x7f86c40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 -- 172.21.15.22:0/443602118 >> 172.21.15.22:0/443602118 conn(0x7f86c40cb2a0 msgr2=0x7f86c40cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 -- 172.21.15.22:0/443602118 shutdown_connections 2023-06-23T18:04:48.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.766+0000 7f86caf3c700 1 -- 172.21.15.22:0/443602118 wait complete. 2023-06-23T18:04:48.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.770+0000 7f86caf3c700 1 Processor -- start 2023-06-23T18:04:48.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.770+0000 7f86caf3c700 1 -- start start 2023-06-23T18:04:48.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.770+0000 7f86caf3c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.770+0000 7f86caf3c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40cd160 0x7f86c41e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 0x7f86c40c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b26d0 con 0x7f86c40c9c60 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4150190 con 0x7f86c40cd160 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b2f20 con 0x7f86c4157f00 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41934/0 (socket says 172.21.15.22:41934) 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 -- 172.21.15.22:0/1689164489 learned_addr learned my addr 172.21.15.22:0/1689164489 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9739700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40cd160 0x7f86c41e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ca73b700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 0x7f86c40c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 -- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 msgr2=0x7f86c40c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 0x7f86c40c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 -- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40cd160 msgr2=0x7f86c41e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40cd160 0x7f86c41e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 -- 172.21.15.22:0/1689164489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009a70 con 0x7f86c40c9c60 2023-06-23T18:04:48.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ca73b700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 0x7f86c40c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:48.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86c9f3a700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c510 tx=0x7f86c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004ca0 con 0x7f86c40c9c60 2023-06-23T18:04:48.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0004e20 con 0x7f86c40c9c60 2023-06-23T18:04:48.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40b2920 con 0x7f86c40c9c60 2023-06-23T18:04:48.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0007310 con 0x7f86c40c9c60 2023-06-23T18:04:48.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c40b31e0 con 0x7f86c40c9c60 2023-06-23T18:04:48.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c404e4c0 con 0x7f86c40c9c60 2023-06-23T18:04:48.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.774+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f86c0007470 con 0x7f86c40c9c60 2023-06-23T18:04:48.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.778+0000 7f86ba7fc700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 0x7f86a80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:48.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.778+0000 7f86c9739700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 0x7f86a80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:48.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.778+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f86c0083e10 con 0x7f86c40c9c60 2023-06-23T18:04:48.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.778+0000 7f86c9739700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 0x7f86a80687a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b0830 tx=0x7f86b400b4f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:48.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.786+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f86c0050ee0 con 0x7f86c40c9c60 2023-06-23T18:04:48.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.954+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c40c3c20 con 0x7f86c40c9c60 2023-06-23T18:04:48.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.954+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1689164489 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f86c0007720 con 0x7f86c40c9c60 2023-06-23T18:04:48.969 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:48.969 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:48.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 msgr2=0x7f86a80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 0x7f86a80687a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b0830 tx=0x7f86b400b4f0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 msgr2=0x7f86c414fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c510 tx=0x7f86c000c8d0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 shutdown_connections 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c4157f00 0x7f86c40c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f86a80662f0 0x7f86a80687a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c40cd160 0x7f86c41e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 --2- 172.21.15.22:0/1689164489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40c9c60 0x7f86c414fc50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 >> 172.21.15.22:0/1689164489 conn(0x7f86c40cb2a0 msgr2=0x7f86c40cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 shutdown_connections 2023-06-23T18:04:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:48.958+0000 7f86caf3c700 1 -- 172.21.15.22:0/1689164489 wait complete. 2023-06-23T18:04:48.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:03:36.275123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:03:36.707871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-23T18:03:39.789766+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-23T18:04:48.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:04:49.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.094+0000 7f9a108d9700 1 Processor -- start 2023-06-23T18:04:49.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.094+0000 7f9a108d9700 1 -- start start 2023-06-23T18:04:49.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9a0c0707b0 0x7f9a0c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0cbaa0 con 0x7f9a0c069960 2023-06-23T18:04:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0cbd80 con 0x7f9a0c070c10 2023-06-23T18:04:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c071110 con 0x7f9a0c0707b0 2023-06-23T18:04:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0ad9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9a0c069960 0x7f9a0c069d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42666/0 (socket says 172.21.15.22:42666) 2023-06-23T18:04:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0ad9d700 1 -- 172.21.15.22:0/2128234676 learned_addr learned my addr 172.21.15.22:0/2128234676 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0a59c700 1 --2- 172.21.15.22:0/2128234676 >> v2:172.21.15.22:3300/0 conn(0x7f9a0c0707b0 0x7f9a0c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828052627 0 0) 0x7f9a0c0cbaa0 con 0x7f9a0c069960 2023-06-23T18:04:49.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f80035d0 con 0x7f9a0c069960 2023-06-23T18:04:49.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0a59c700 1 -- 172.21.15.22:0/2128234676 >> v1:172.21.15.169:6789/0 conn(0x7f9a0c070c10 legacy=0x7f9a0c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037406917 0 0) 0x7f9a0c0cbd80 con 0x7f9a0c070c10 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0a59c700 1 -- 172.21.15.22:0/2128234676 >> v1:172.21.15.22:6789/0 conn(0x7f9a0c069960 legacy=0x7f9a0c069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0a59c700 1 -- 172.21.15.22:0/2128234676 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cc060 con 0x7f9a0c0707b0 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a0a59c700 1 --2- 172.21.15.22:0/2128234676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0707b0 0x7f9a0c0c9710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009850 tx=0x7f99f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4856bacd4aaa370e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0019f0 con 0x7f9a0c0707b0 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4010590 con 0x7f9a0c0707b0 2023-06-23T18:04:49.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a09d9b700 1 -- 172.21.15.22:0/2128234676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4010940 con 0x7f9a0c0707b0 2023-06-23T18:04:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- 172.21.15.22:0/2128234676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0707b0 msgr2=0x7f9a0c0c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 --2- 172.21.15.22:0/2128234676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0707b0 0x7f9a0c0c9710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009850 tx=0x7f99f4009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- 172.21.15.22:0/2128234676 shutdown_connections 2023-06-23T18:04:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 --2- 172.21.15.22:0/2128234676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0707b0 0x7f9a0c0c9710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- 172.21.15.22:0/2128234676 >> 172.21.15.22:0/2128234676 conn(0x7f9a0c06b700 msgr2=0x7f9a0c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- 172.21.15.22:0/2128234676 shutdown_connections 2023-06-23T18:04:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.098+0000 7f9a108d9700 1 -- 172.21.15.22:0/2128234676 wait complete. 2023-06-23T18:04:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 Processor -- start 2023-06-23T18:04:49.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- start start 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 0x7f9a0c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 0x7f9a0c0b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15e4d0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ad130 con 0x7f9a0c070c10 2023-06-23T18:04:49.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1502e0 con 0x7f9a0c069960 2023-06-23T18:04:49.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41960/0 (socket says 172.21.15.22:41960) 2023-06-23T18:04:49.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 -- 172.21.15.22:0/1570975175 learned_addr learned my addr 172.21.15.22:0/1570975175 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:49.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0ad9d700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 0x7f9a0c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0a59c700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 0x7f9a0c0b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 -- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 msgr2=0x7f9a0c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 0x7f9a0c0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 -- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 msgr2=0x7f9a0c0b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 0x7f9a0c0b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 -- 172.21.15.22:0/1570975175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40094b0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0ad9d700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 0x7f9a0c0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:49.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0a59c700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 0x7f9a0c0b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a0b59e700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c4d0 tx=0x7f9a0400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040054e0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040036a0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c15db30 con 0x7f9a0c0ac780 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04003a50 con 0x7f9a0c0ac780 2023-06-23T18:04:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.102+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c15e0f0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.106+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c04e4c0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.106+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9a04003bb0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.106+0000 7f9a037fe700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 0x7f99ec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.110+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9a04083480 con 0x7f9a0c0ac780 2023-06-23T18:04:49.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.110+0000 7f9a0ad9d700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 0x7f99ec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.110+0000 7f9a0ad9d700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 0x7f99ec0687a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00d340 tx=0x7f99fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.114+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f9a04050660 con 0x7f9a0c0ac780 2023-06-23T18:04:49.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:04:49.238 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:04:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.274+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a0c0b18d0 con 0x7f9a0c0ac780 2023-06-23T18:04:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.274+0000 7f9a037fe700 1 -- 172.21.15.22:0/1570975175 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a04003e60 con 0x7f9a0c0ac780 2023-06-23T18:04:49.289 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:04:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 msgr2=0x7f99ec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 0x7f99ec0687a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00d340 tx=0x7f99fc009040 comp rx=0 tx=0).stop 2023-06-23T18:04:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 msgr2=0x7f9a0c0acbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c4d0 tx=0x7f9a0400c890 comp rx=0 tx=0).stop 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 shutdown_connections 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a0c0ac780 0x7f9a0c0acbf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f99ec0662f0 0x7f99ec0687a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a0c070c10 0x7f9a0c0b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 --2- 172.21.15.22:0/1570975175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a0c069960 0x7f9a0c0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 >> 172.21.15.22:0/1570975175 conn(0x7f9a0c06b700 msgr2=0x7f9a0c0cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 shutdown_connections 2023-06-23T18:04:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.282+0000 7f9a108d9700 1 -- 172.21.15.22:0/1570975175 wait complete. 2023-06-23T18:04:49.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:04:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.418+0000 7fe036df7700 1 Processor -- start 2023-06-23T18:04:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.418+0000 7fe036df7700 1 -- start start 2023-06-23T18:04:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe030070e50 0x7fe03006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03006eab0 con 0x7fe030158b20 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0300cd680 con 0x7fe030158ef0 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300cdf20 con 0x7fe030070e50 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe0365f6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe030070e50 0x7fe03006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe035df5700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe030158b20 0x7fe030070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42682/0 (socket says 172.21.15.22:42682) 2023-06-23T18:04:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe035df5700 1 -- 172.21.15.22:0/1790836596 learned_addr learned my addr 172.21.15.22:0/1790836596 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe0365f6700 1 -- 172.21.15.22:0/1790836596 >> v1:172.21.15.169:6789/0 conn(0x7fe030158ef0 legacy=0x7fe0300c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe0365f6700 1 -- 172.21.15.22:0/1790836596 >> v1:172.21.15.22:6789/0 conn(0x7fe030158b20 legacy=0x7fe030070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe0365f6700 1 -- 172.21.15.22:0/1790836596 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300cd960 con 0x7fe030070e50 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe0365f6700 1 --2- 172.21.15.22:0/1790836596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe03006e440 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe02800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd2f91668765c8d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe034df3700 1 -- 172.21.15.22:0/1790836596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02800e070 con 0x7fe030070e50 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe034df3700 1 -- 172.21.15.22:0/1790836596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02800bb90 con 0x7fe030070e50 2023-06-23T18:04:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe034df3700 1 -- 172.21.15.22:0/1790836596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028003a40 con 0x7fe030070e50 2023-06-23T18:04:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- 172.21.15.22:0/1790836596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 msgr2=0x7fe03006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 --2- 172.21.15.22:0/1790836596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe03006e440 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe02800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- 172.21.15.22:0/1790836596 shutdown_connections 2023-06-23T18:04:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 --2- 172.21.15.22:0/1790836596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe03006e440 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- 172.21.15.22:0/1790836596 >> 172.21.15.22:0/1790836596 conn(0x7fe03006b700 msgr2=0x7fe0300b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- 172.21.15.22:0/1790836596 shutdown_connections 2023-06-23T18:04:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 -- 172.21.15.22:0/1790836596 wait complete. 2023-06-23T18:04:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.422+0000 7fe036df7700 1 Processor -- start 2023-06-23T18:04:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- start start 2023-06-23T18:04:49.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe030158b20 0x7fe0300c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 0x7fe030153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0301f5c10 con 0x7fe030070e50 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c8c80 con 0x7fe030158ef0 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c6690 con 0x7fe030158b20 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41986/0 (socket says 172.21.15.22:41986) 2023-06-23T18:04:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 -- 172.21.15.22:0/1209427003 learned_addr learned my addr 172.21.15.22:0/1209427003 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0355f4700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe030158b20 0x7fe0300c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0365f6700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 0x7fe030153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 -- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe030158b20 msgr2=0x7fe0300c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe030158b20 0x7fe0300c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 -- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 msgr2=0x7fe030153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 0x7fe030153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 -- 172.21.15.22:0/1209427003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe028009a70 con 0x7fe030070e50 2023-06-23T18:04:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0365f6700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 0x7fe030153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:04:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe035df5700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c500 tx=0x7fe02c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c005d90 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c0070d0 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300c6970 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c007460 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0300c6f30 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.426+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03004e4c0 con 0x7fe030070e50 2023-06-23T18:04:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.430+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe02c0075c0 con 0x7fe030070e50 2023-06-23T18:04:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.430+0000 7fe0267fc700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 0x7fe0140687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.430+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe02c083590 con 0x7fe030070e50 2023-06-23T18:04:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.430+0000 7fe0355f4700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 0x7fe0140687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.430+0000 7fe0355f4700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 0x7fe0140687a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0301f4c20 tx=0x7fe020008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.438+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe02c0506e0 con 0x7fe030070e50 2023-06-23T18:04:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.602+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe03006bf40 con 0x7fe030070e50 2023-06-23T18:04:49.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.602+0000 7fe0267fc700 1 -- 172.21.15.22:0/1209427003 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe02c053d00 con 0x7fe030070e50 2023-06-23T18:04:49.617 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:04:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 msgr2=0x7fe0140687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 0x7fe0140687a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0301f4c20 tx=0x7fe020008040 comp rx=0 tx=0).stop 2023-06-23T18:04:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 msgr2=0x7fe0301f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c500 tx=0x7fe02c00c8c0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 shutdown_connections 2023-06-23T18:04:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe0140662f0 0x7fe0140687a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe030158ef0 0x7fe030153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe030158b20 0x7fe0300c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 --2- 172.21.15.22:0/1209427003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe030070e50 0x7fe0301f56d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 >> 172.21.15.22:0/1209427003 conn(0x7fe03006b700 msgr2=0x7fe0300cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 shutdown_connections 2023-06-23T18:04:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.606+0000 7fe036df7700 1 -- 172.21.15.22:0/1209427003 wait complete. 2023-06-23T18:04:49.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:04:49.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 Processor -- start 2023-06-23T18:04:49.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 -- start start 2023-06-23T18:04:49.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 0x7f23ac0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac0ccb30 con 0x7f23ac0d0730 2023-06-23T18:04:49.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac0cfcb0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23abfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0d2880 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23ab7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f23ac0d0730 0x7f23ac0ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42690/0 (socket says 172.21.15.22:42690) 2023-06-23T18:04:49.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23ab7fe700 1 -- 172.21.15.22:0/2912734045 learned_addr learned my addr 172.21.15.22:0/2912734045 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:49.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.739+0000 7f23aaffd700 1 --2- 172.21.15.22:0/2912734045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 0x7f23ac0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916603063 0 0) 0x7f23ac0ccb30 con 0x7f23ac0d0730 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23940035d0 con 0x7f23ac0d0730 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23aaffd700 1 -- 172.21.15.22:0/2912734045 >> v1:172.21.15.169:6789/0 conn(0x7f23ac0d1f30 legacy=0x7f23ac0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058910944 0 0) 0x7f23ac0cfcb0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23aaffd700 1 -- 172.21.15.22:0/2912734045 >> v1:172.21.15.22:6789/0 conn(0x7f23ac0d0730 legacy=0x7f23ac0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23aaffd700 1 -- 172.21.15.22:0/2912734045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0d2b60 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23aaffd700 1 --2- 172.21.15.22:0/2912734045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 0x7f23ac0cf770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009870 tx=0x7f23a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f501ca283cfaf6a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40019f0 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a0004030 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23a9ffb700 1 -- 172.21.15.22:0/2912734045 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00043a0 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 -- 172.21.15.22:0/2912734045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 msgr2=0x7f23ac0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 --2- 172.21.15.22:0/2912734045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 0x7f23ac0cf770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009870 tx=0x7f23a0009b80 comp rx=0 tx=0).stop 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 -- 172.21.15.22:0/2912734045 shutdown_connections 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 --2- 172.21.15.22:0/2912734045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0cf3a0 0x7f23ac0cf770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 -- 172.21.15.22:0/2912734045 >> 172.21.15.22:0/2912734045 conn(0x7f23ac06a280 msgr2=0x7f23ac09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 -- 172.21.15.22:0/2912734045 shutdown_connections 2023-06-23T18:04:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.743+0000 7f23abfff700 1 -- 172.21.15.22:0/2912734045 wait complete. 2023-06-23T18:04:49.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.747+0000 7f23abfff700 1 Processor -- start 2023-06-23T18:04:49.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.747+0000 7f23abfff700 1 -- start start 2023-06-23T18:04:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 0x7f23ac0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0d0730 0x7f23ac15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 0x7f23ac0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac1f20d0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0cc010 con 0x7f23ac0cf3a0 2023-06-23T18:04:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac1f1190 con 0x7f23ac0d0730 2023-06-23T18:04:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23aaffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 0x7f23ac0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 0x7f23ac0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23aaffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 0x7f23ac0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:45390/0 (socket says 172.21.15.22:45390) 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23aaffd700 1 -- 172.21.15.22:0/2210741585 learned_addr learned my addr 172.21.15.22:0/2210741585 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23aa7fc700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0d0730 0x7f23ac15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 -- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0d0730 msgr2=0x7f23ac15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0d0730 0x7f23ac15e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 -- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 msgr2=0x7f23ac0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 0x7f23ac0b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 -- 172.21.15.22:0/2210741585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0009450 con 0x7f23ac0d1f30 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23ab7fe700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 0x7f23ac0b3bb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23a4001fd0 tx=0x7f23a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400ad80 con 0x7f23ac0d1f30 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40070d0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4007480 con 0x7f23ac0d1f30 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac1f1450 con 0x7f23ac0d1f30 2023-06-23T18:04:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.751+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac1f1a10 con 0x7f23ac0d1f30 2023-06-23T18:04:49.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac04e4c0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f23a40075e0 con 0x7f23ac0d1f30 2023-06-23T18:04:49.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23937fe700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 0x7f23980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:49.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f23a4083c50 con 0x7f23ac0d1f30 2023-06-23T18:04:49.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23aaffd700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 0x7f23980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:49.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.755+0000 7f23aaffd700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 0x7f23980687a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009bc0 tx=0x7f23a0008000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:49.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.763+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f23a4089020 con 0x7f23ac0d1f30 2023-06-23T18:04:49.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.927+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23ac06a680 con 0x7f23ac0d1f30 2023-06-23T18:04:49.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.927+0000 7f23937fe700 1 -- 172.21.15.22:0/2210741585 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23a4050d20 con 0x7f23ac0d1f30 2023-06-23T18:04:49.939 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 msgr2=0x7f23980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 0x7f23980687a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009bc0 tx=0x7f23a0008000 comp rx=0 tx=0).stop 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 msgr2=0x7f23ac0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 0x7f23ac0b3bb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23a4001fd0 tx=0x7f23a400a340 comp rx=0 tx=0).stop 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 shutdown_connections 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23ac0d1f30 0x7f23ac0b3bb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23ac0d0730 0x7f23ac15e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f23980662f0 0x7f23980687a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 --2- 172.21.15.22:0/2210741585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0cf3a0 0x7f23ac0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:49.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 >> 172.21.15.22:0/2210741585 conn(0x7f23ac06a280 msgr2=0x7f23ac0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:49.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 shutdown_connections 2023-06-23T18:04:49.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:49.931+0000 7f23abfff700 1 -- 172.21.15.22:0/2210741585 wait complete. 2023-06-23T18:04:49.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:04:50.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 Processor -- start 2023-06-23T18:04:50.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 -- start start 2023-06-23T18:04:50.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b800cd8e0 0x7f3b800cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8015a700 con 0x7f3b800b1d60 2023-06-23T18:04:50.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b800ce7b0 con 0x7f3b80159c30 2023-06-23T18:04:50.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85b42700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b800cea90 con 0x7f3b800cd8e0 2023-06-23T18:04:50.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85341700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3b800b1d60 0x7f3b8006f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42700/0 (socket says 172.21.15.22:42700) 2023-06-23T18:04:50.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b85341700 1 -- 172.21.15.22:0/1214343633 learned_addr learned my addr 172.21.15.22:0/1214343633 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:50.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7ffff700 1 --2- 172.21.15.22:0/1214343633 >> v2:172.21.15.169:3300/0 conn(0x7f3b800cd8e0 0x7f3b800cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7f7fe700 1 -- 172.21.15.22:0/1214343633 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348803356 0 0) 0x7f3b8015a700 con 0x7f3b800b1d60 2023-06-23T18:04:50.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7f7fe700 1 -- 172.21.15.22:0/1214343633 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b6c0035d0 con 0x7f3b800b1d60 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7ffff700 1 -- 172.21.15.22:0/1214343633 >> v1:172.21.15.169:6789/0 conn(0x7f3b80159c30 legacy=0x7f3b8015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7ffff700 1 -- 172.21.15.22:0/1214343633 >> v1:172.21.15.22:6789/0 conn(0x7f3b800b1d60 legacy=0x7f3b8006f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7ffff700 1 -- 172.21.15.22:0/1214343633 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8015a390 con 0x7f3b800cd8e0 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.063+0000 7f3b7ffff700 1 --2- 172.21.15.22:0/1214343633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800cd8e0 0x7f3b800cdeb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009850 tx=0x7f3b68009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91233dcefcd5fd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b7f7fe700 1 -- 172.21.15.22:0/1214343633 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011830 con 0x7f3b800cd8e0 2023-06-23T18:04:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b7f7fe700 1 -- 172.21.15.22:0/1214343633 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68011990 con 0x7f3b800cd8e0 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b7f7fe700 1 -- 172.21.15.22:0/1214343633 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011d00 con 0x7f3b800cd8e0 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- 172.21.15.22:0/1214343633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800cd8e0 msgr2=0x7f3b800cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 --2- 172.21.15.22:0/1214343633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800cd8e0 0x7f3b800cdeb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009850 tx=0x7f3b68009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- 172.21.15.22:0/1214343633 shutdown_connections 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 --2- 172.21.15.22:0/1214343633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800cd8e0 0x7f3b800cdeb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- 172.21.15.22:0/1214343633 >> 172.21.15.22:0/1214343633 conn(0x7f3b800ccd60 msgr2=0x7f3b800cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- 172.21.15.22:0/1214343633 shutdown_connections 2023-06-23T18:04:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- 172.21.15.22:0/1214343633 wait complete. 2023-06-23T18:04:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 Processor -- start 2023-06-23T18:04:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.067+0000 7f3b85b42700 1 -- start start 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800b1d60 0x7f3b801f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b800cd8e0 0x7f3b800cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c6790 con 0x7f3b80159c30 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c5a70 con 0x7f3b800b1d60 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c5d50 con 0x7f3b800cd8e0 2023-06-23T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42004/0 (socket says 172.21.15.22:42004) 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 -- 172.21.15.22:0/3724536273 learned_addr learned my addr 172.21.15.22:0/3724536273 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b7ffff700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b800cd8e0 0x7f3b800cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b84b40700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800b1d60 0x7f3b801f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 -- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b800cd8e0 msgr2=0x7f3b800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b800cd8e0 0x7f3b800cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 -- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800b1d60 msgr2=0x7f3b801f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800b1d60 0x7f3b801f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 -- 172.21.15.22:0/3724536273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b680094b0 con 0x7f3b80159c30 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85341700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800c4f0 tx=0x7f3b7800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78005d90 con 0x7f3b80159c30 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b780070d0 con 0x7f3b80159c30 2023-06-23T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8006ab10 con 0x7f3b80159c30 2023-06-23T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78007460 con 0x7f3b80159c30 2023-06-23T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8006b0d0 con 0x7f3b80159c30 2023-06-23T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.071+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8006b2c0 con 0x7f3b80159c30 2023-06-23T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.075+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3b780075c0 con 0x7f3b80159c30 2023-06-23T18:04:50.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.075+0000 7f3b7d7fa700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 0x7f3b600687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.075+0000 7f3b84b40700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 0x7f3b600687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.075+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b78083510 con 0x7f3b80159c30 2023-06-23T18:04:50.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.075+0000 7f3b84b40700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 0x7f3b600687a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b8006cc50 tx=0x7f3b70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.083+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3b780505e0 con 0x7f3b80159c30 2023-06-23T18:04:50.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.239+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b8004e4c0 con 0x7f3b80159c30 2023-06-23T18:04:50.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b7d7fa700 1 -- 172.21.15.22:0/3724536273 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b78053c00 con 0x7f3b80159c30 2023-06-23T18:04:50.253 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-06-23T18:04:50.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 msgr2=0x7f3b600687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 0x7f3b600687a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b8006cc50 tx=0x7f3b70009040 comp rx=0 tx=0).stop 2023-06-23T18:04:50.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 msgr2=0x7f3b800cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800c4f0 tx=0x7f3b7800c800 comp rx=0 tx=0).stop 2023-06-23T18:04:50.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 shutdown_connections 2023-06-23T18:04:50.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b80159c30 0x7f3b800cfad0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b800cd8e0 0x7f3b800cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3b600662f0 0x7f3b600687a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 --2- 172.21.15.22:0/3724536273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b800b1d60 0x7f3b801f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 >> 172.21.15.22:0/3724536273 conn(0x7f3b800ccd60 msgr2=0x7f3b800b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 shutdown_connections 2023-06-23T18:04:50.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.243+0000 7f3b85b42700 1 -- 172.21.15.22:0/3724536273 wait complete. 2023-06-23T18:04:50.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-23T18:04:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.367+0000 7f47a2c68700 1 Processor -- start 2023-06-23T18:04:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.367+0000 7f47a2c68700 1 -- start start 2023-06-23T18:04:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f479c0b04e0 0x7f479c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f479c069960 0x7f479c069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0ce750 con 0x7f479c0b04e0 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c160500 con 0x7f479c069960 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c06b240 con 0x7f479c0cefa0 2023-06-23T18:04:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f479c0b04e0 0x7f479c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f479c0b04e0 0x7f479c0b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42014/0 (socket says 172.21.15.22:42014) 2023-06-23T18:04:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 -- 172.21.15.22:0/1083810075 learned_addr learned my addr 172.21.15.22:0/1083810075 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:50.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2467700 1 --2- 172.21.15.22:0/1083810075 >> v2:172.21.15.169:3300/0 conn(0x7f479c069960 0x7f479c069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1465700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 -- 172.21.15.22:0/1083810075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 msgr2=0x7f479c06aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c06aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 -- 172.21.15.22:0/1083810075 >> v2:172.21.15.169:3300/0 conn(0x7f479c069960 msgr2=0x7f479c069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 --2- 172.21.15.22:0/1083810075 >> v2:172.21.15.169:3300/0 conn(0x7f479c069960 0x7f479c069dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 -- 172.21.15.22:0/1083810075 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0b0df0 con 0x7f479c0b04e0 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a1c66700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c0b08b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f479c073320 tx=0x7f4798009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac17ea63fc14082 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a0c64700 1 -- 172.21.15.22:0/1083810075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798011830 con 0x7f479c0b04e0 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a0c64700 1 -- 172.21.15.22:0/1083810075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798011990 con 0x7f479c0b04e0 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a0c64700 1 -- 172.21.15.22:0/1083810075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798011d20 con 0x7f479c0b04e0 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- 172.21.15.22:0/1083810075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 msgr2=0x7f479c0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c0b08b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f479c073320 tx=0x7f4798009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- 172.21.15.22:0/1083810075 shutdown_connections 2023-06-23T18:04:50.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- 172.21.15.22:0/1083810075 >> v2:172.21.15.169:3300/0 conn(0x7f479c069960 0x7f479c069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c06aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 --2- 172.21.15.22:0/1083810075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c0b08b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- 172.21.15.22:0/1083810075 >> 172.21.15.22:0/1083810075 conn(0x7f479c06ca60 msgr2=0x7f479c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- 172.21.15.22:0/1083810075 shutdown_connections 2023-06-23T18:04:50.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.371+0000 7f47a2c68700 1 -- 172.21.15.22:0/1083810075 wait complete. 2023-06-23T18:04:50.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 Processor -- start 2023-06-23T18:04:50.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- start start 2023-06-23T18:04:50.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 0x7f479c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c1f9b70 con 0x7f479c0b04e0 2023-06-23T18:04:50.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c14d7d0 con 0x7f479c150340 2023-06-23T18:04:50.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c1f8f60 con 0x7f479c0cefa0 2023-06-23T18:04:50.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42020/0 (socket says 172.21.15.22:42020) 2023-06-23T18:04:50.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 -- 172.21.15.22:0/3022341610 learned_addr learned my addr 172.21.15.22:0/3022341610 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:50.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1465700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2467700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 0x7f479c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 -- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 msgr2=0x7f479c149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 -- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 msgr2=0x7f479c0c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 0x7f479c0c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 -- 172.21.15.22:0/3022341610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009570 con 0x7f479c0b04e0 2023-06-23T18:04:50.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2467700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 0x7f479c0c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:50.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a1c66700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f479c061110 tx=0x7f4798004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004510 con 0x7f479c0b04e0 2023-06-23T18:04:50.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479801e070 con 0x7f479c0b04e0 2023-06-23T18:04:50.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c1f9240 con 0x7f479c0b04e0 2023-06-23T18:04:50.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798003900 con 0x7f479c0b04e0 2023-06-23T18:04:50.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.375+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c1f9800 con 0x7f479c0b04e0 2023-06-23T18:04:50.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c04e4c0 con 0x7f479c0b04e0 2023-06-23T18:04:50.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4798003a60 con 0x7f479c0b04e0 2023-06-23T18:04:50.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47927fc700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 0x7f47800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4798084f70 con 0x7f479c0b04e0 2023-06-23T18:04:50.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47a1465700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 0x7f47800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.379+0000 7f47a1465700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 0x7f47800687a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f479c0c86d0 tx=0x7f478c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.387+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f47980b5700 con 0x7f479c0b04e0 2023-06-23T18:04:50.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f479c060890 con 0x7f479c0b04e0 2023-06-23T18:04:50.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47927fc700 1 -- 172.21.15.22:0/3022341610 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4798020020 con 0x7f479c0b04e0 2023-06-23T18:04:50.558 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-06-23T18:04:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 msgr2=0x7f47800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 0x7f47800687a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f479c0c86d0 tx=0x7f478c008040 comp rx=0 tx=0).stop 2023-06-23T18:04:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 msgr2=0x7f479c14d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f479c061110 tx=0x7f4798004770 comp rx=0 tx=0).stop 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 shutdown_connections 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47800662f0 0x7f47800687a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f479c150340 0x7f479c0c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f479c0cefa0 0x7f479c149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 --2- 172.21.15.22:0/3022341610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479c0b04e0 0x7f479c14d290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 >> 172.21.15.22:0/3022341610 conn(0x7f479c069960 msgr2=0x7f479c0c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 shutdown_connections 2023-06-23T18:04:50.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.547+0000 7f47a2c68700 1 -- 172.21.15.22:0/3022341610 wait complete. 2023-06-23T18:04:50.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 Processor -- start 2023-06-23T18:04:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 -- start start 2023-06-23T18:04:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f61c80d0e40 0x7f61c80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61c80b2560 0x7f61c815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c80cd7e0 con 0x7f61c80cf630 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c80d0a20 con 0x7f61c80b2560 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce956700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c80cd500 con 0x7f61c80d0e40 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.671+0000 7f61ce155700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61c80b2560 0x7f61c815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f61c80b2560 0x7f61c815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42024/0 (socket says 172.21.15.22:42024) 2023-06-23T18:04:50.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 -- 172.21.15.22:0/583111092 learned_addr learned my addr 172.21.15.22:0/583111092 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61cd954700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61c80cf630 0x7f61c80cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42710/0 (socket says 172.21.15.22:42710) 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61cd153700 1 --2- 172.21.15.22:0/583111092 >> v2:172.21.15.169:3300/0 conn(0x7f61c80d0e40 0x7f61c80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 -- 172.21.15.22:0/583111092 >> v2:172.21.15.169:3300/0 conn(0x7f61c80d0e40 msgr2=0x7f61c80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 --2- 172.21.15.22:0/583111092 >> v2:172.21.15.169:3300/0 conn(0x7f61c80d0e40 0x7f61c80cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 -- 172.21.15.22:0/583111092 >> v1:172.21.15.22:6789/0 conn(0x7f61c80cf630 legacy=0x7f61c80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 -- 172.21.15.22:0/583111092 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80d0360 con 0x7f61c80b2560 2023-06-23T18:04:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce155700 1 --2- 172.21.15.22:0/583111092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80b2560 0x7f61c815fa30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f61c8070460 tx=0x7f61c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e325a59decc896b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61cc952700 1 -- 172.21.15.22:0/583111092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000e070 con 0x7f61c80b2560 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61cc952700 1 -- 172.21.15.22:0/583111092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c000b9c0 con 0x7f61c80b2560 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61cc952700 1 -- 172.21.15.22:0/583111092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000bd70 con 0x7f61c80b2560 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- 172.21.15.22:0/583111092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80b2560 msgr2=0x7f61c815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 --2- 172.21.15.22:0/583111092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80b2560 0x7f61c815fa30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f61c8070460 tx=0x7f61c000b330 comp rx=0 tx=0).stop 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- 172.21.15.22:0/583111092 shutdown_connections 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 --2- 172.21.15.22:0/583111092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80b2560 0x7f61c815fa30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 --2- 172.21.15.22:0/583111092 >> v2:172.21.15.169:3300/0 conn(0x7f61c80d0e40 0x7f61c80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- 172.21.15.22:0/583111092 >> 172.21.15.22:0/583111092 conn(0x7f61c8069960 msgr2=0x7f61c8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- 172.21.15.22:0/583111092 shutdown_connections 2023-06-23T18:04:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- 172.21.15.22:0/583111092 wait complete. 2023-06-23T18:04:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 Processor -- start 2023-06-23T18:04:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.675+0000 7f61ce956700 1 -- start start 2023-06-23T18:04:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c80b2560 0x7f61c806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 0x7f61c81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c3dd0 con 0x7f61c80d0e40 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c3fe0 con 0x7f61c80b2560 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c42c0 con 0x7f61c80cf630 2023-06-23T18:04:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42040/0 (socket says 172.21.15.22:42040) 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 -- 172.21.15.22:0/26898803 learned_addr learned my addr 172.21.15.22:0/26898803 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61cd153700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 0x7f61c81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61cd954700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c80b2560 0x7f61c806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 -- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 msgr2=0x7f61c81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 0x7f61c81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 -- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c80b2560 msgr2=0x7f61c806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c80b2560 0x7f61c806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 -- 172.21.15.22:0/26898803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0009b30 con 0x7f61c80d0e40 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61cd153700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 0x7f61c81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce155700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f61c000b370 tx=0x7f61c0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00099b0 con 0x7f61c80d0e40 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00092e0 con 0x7f61c80d0e40 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c80c45a0 con 0x7f61c80d0e40 2023-06-23T18:04:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c001c640 con 0x7f61c80d0e40 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.679+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c8153710 con 0x7f61c80d0e40 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c80c4a60 con 0x7f61c80d0e40 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f61c001c7a0 con 0x7f61c80d0e40 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61be7fc700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 0x7f61ac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f61c0085a00 con 0x7f61c80d0e40 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61cd954700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 0x7f61ac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.683+0000 7f61cd954700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 0x7f61ac0687a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000f60 tx=0x7f61b8007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:50.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.691+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f61c0052b60 con 0x7f61c80d0e40 2023-06-23T18:04:50.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61c804e4c0 con 0x7f61c80d0e40 2023-06-23T18:04:50.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61be7fc700 1 -- 172.21.15.22:0/26898803 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f61c0056180 con 0x7f61c80d0e40 2023-06-23T18:04:50.903 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:50.903 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 msgr2=0x7f61ac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 0x7f61ac0687a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000f60 tx=0x7f61b8007d30 comp rx=0 tx=0).stop 2023-06-23T18:04:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 msgr2=0x7f61c8154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.891+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f61c000b370 tx=0x7f61c0004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 shutdown_connections 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c80d0e40 0x7f61c8154910 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c80cf630 0x7f61c81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61ac0662f0 0x7f61ac0687a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 --2- 172.21.15.22:0/26898803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c80b2560 0x7f61c806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:50.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 >> 172.21.15.22:0/26898803 conn(0x7f61c8069960 msgr2=0x7f61c80b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 shutdown_connections 2023-06-23T18:04:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:50.895+0000 7f61ce956700 1 -- 172.21.15.22:0/26898803 wait complete. 2023-06-23T18:04:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:50.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.019+0000 7f7fa2a7e700 1 Processor -- start 2023-06-23T18:04:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.019+0000 7f7fa2a7e700 1 -- start start 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa2a7e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7f9c159c30 0x7f7f9c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa2a7e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0b0420 con 0x7f7f9c0affd0 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa2a7e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c06d0d0 con 0x7f7f9c071450 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa2a7e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c06d560 con 0x7f7f9c159c30 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7f9c0affd0 0x7f7f9c070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42716/0 (socket says 172.21.15.22:42716) 2023-06-23T18:04:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 -- 172.21.15.22:0/2287549444 learned_addr learned my addr 172.21.15.22:0/2287549444 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2287549444 >> v2:172.21.15.169:3300/0 conn(0x7f7f9c159c30 0x7f7f9c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934434138 0 0) 0x7f7f98001110 con 0x7f7f9c0affd0 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f880035d0 con 0x7f7f9c0affd0 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022014454 0 0) 0x7f7f880035d0 con 0x7f7f9c0affd0 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 >> v2:172.21.15.169:3300/0 conn(0x7f7f9c159c30 msgr2=0x7f7f9c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 --2- 172.21.15.22:0/2287549444 >> v2:172.21.15.169:3300/0 conn(0x7f7f9c159c30 0x7f7f9c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 >> v1:172.21.15.169:6789/0 conn(0x7f7f9c071450 legacy=0x7f7f9c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c06c500 con 0x7f7f9c0affd0 2023-06-23T18:04:51.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f7f98001800 con 0x7f7f9c0affd0 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 >> v1:172.21.15.22:6789/0 conn(0x7f7f9c0affd0 legacy=0x7f7f9c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f88003a20 0x7f7f88003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 0x7f7f88004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88005220 0x7f7f8800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c06d560 con 0x7f7f88004870 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8800c130 con 0x7f7f88003a20 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8800c2f0 con 0x7f7f88005220 2023-06-23T18:04:51.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 0x7f7f88004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f88003a20 0x7f7f88003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa227d700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88005220 0x7f7f8800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 -- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88005220 msgr2=0x7f7f8800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88005220 0x7f7f8800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 -- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f88003a20 msgr2=0x7f7f88003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f88003a20 0x7f7f88003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa227d700 1 -- 172.21.15.22:0/2287549444 reap_dead start 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 -- 172.21.15.22:0/2287549444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880035d0 con 0x7f7f88004870 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f88003a20 0x7f7f88003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 0x7f7f88004ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800c240 tx=0x7f7f880058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab877aafe8f6514 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98007db0 con 0x7f7f88004870 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.023+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9800cdc0 con 0x7f7f88004870 2023-06-23T18:04:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa0a7a700 1 -- 172.21.15.22:0/2287549444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98005bc0 con 0x7f7f88004870 2023-06-23T18:04:51.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 msgr2=0x7f7f88004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 0x7f7f88004ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800c240 tx=0x7f7f880058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2287549444 shutdown_connections 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2287549444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f88004870 0x7f7f88004ce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2287549444 >> 172.21.15.22:0/2287549444 conn(0x7f7f9c06a5f0 msgr2=0x7f7f9c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2287549444 shutdown_connections 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2287549444 wait complete. 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 Processor -- start 2023-06-23T18:04:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.027+0000 7f7fa2a7e700 1 -- start start 2023-06-23T18:04:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88004870 0x7f7f9c0c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 0x7f7f9c1f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c8790 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c150f70 con 0x7f7f9c0b3b50 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c2080 con 0x7f7f88004870 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42054/0 (socket says 172.21.15.22:42054) 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 -- 172.21.15.22:0/2135142153 learned_addr learned my addr 172.21.15.22:0/2135142153 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88004870 0x7f7f9c0c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa227d700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 0x7f7f9c1f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 -- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88004870 msgr2=0x7f7f9c0c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88004870 0x7f7f9c0c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 -- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 msgr2=0x7f7f9c1f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 0x7f7f9c1f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 -- 172.21.15.22:0/2135142153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c151210 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa227d700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 0x7f7f9c1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa127b700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c100 tx=0x7f7f9c06d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980017c0 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f98007d50 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c1514f0 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980072f0 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.031+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c1f1080 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c04e4c0 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7f98007450 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7f927fc700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 0x7f7f800687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7f980856b0 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 0x7f7f800687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.035+0000 7f7fa1a7c700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 0x7f7f800687a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c063530 tx=0x7f7f8c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.043+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7f98052970 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.207+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f9c0b0420 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.207+0000 7f7f927fc700 1 -- 172.21.15.22:0/2135142153 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7f7f9c0b0420 con 0x7f7f9c0b36e0 2023-06-23T18:04:51.220 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:51.220 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:51.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 msgr2=0x7f7f800687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 0x7f7f800687a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c063530 tx=0x7f7f8c009240 comp rx=0 tx=0).stop 2023-06-23T18:04:51.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 msgr2=0x7f7f9c1f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c100 tx=0x7f7f9c06d040 comp rx=0 tx=0).stop 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 shutdown_connections 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7f800662f0 0x7f7f800687a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f9c0b3b50 0x7f7f9c1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0b36e0 0x7f7f9c1f0600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 --2- 172.21.15.22:0/2135142153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f88004870 0x7f7f9c0c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 >> 172.21.15.22:0/2135142153 conn(0x7f7f9c06a5f0 msgr2=0x7f7f9c0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 shutdown_connections 2023-06-23T18:04:51.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.211+0000 7f7fa2a7e700 1 -- 172.21.15.22:0/2135142153 wait complete. 2023-06-23T18:04:51.239 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-23T18:04:51.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:51.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 Processor -- start 2023-06-23T18:04:51.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- start start 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec159c30 0x7fb2ec0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec071450 0x7fb2ec0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec0b0420 con 0x7fb2ec0affd0 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec06d560 con 0x7fb2ec071450 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec06d0d0 con 0x7fb2ec159c30 2023-06-23T18:04:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec071450 0x7fb2ec0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2ec071450 0x7fb2ec0cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42058/0 (socket says 172.21.15.22:42058) 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 -- 172.21.15.22:0/3052213570 learned_addr learned my addr 172.21.15.22:0/3052213570 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f08f4700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec159c30 0x7fb2ec0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 -- 172.21.15.22:0/3052213570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec159c30 msgr2=0x7fb2ec0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec159c30 0x7fb2ec0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 -- 172.21.15.22:0/3052213570 >> v1:172.21.15.169:6789/0 conn(0x7fb2ec0affd0 legacy=0x7fb2ec070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 -- 172.21.15.22:0/3052213570 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0718d0 con 0x7fb2ec071450 2023-06-23T18:04:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f10f5700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec0cbbe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009910 tx=0x7fb2e4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739bd2e527f7e406 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2eb7fe700 1 -- 172.21.15.22:0/3052213570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4011830 con 0x7fb2ec071450 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2eb7fe700 1 -- 172.21.15.22:0/3052213570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4011990 con 0x7fb2ec071450 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2eb7fe700 1 -- 172.21.15.22:0/3052213570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4011d40 con 0x7fb2ec071450 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- 172.21.15.22:0/3052213570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 msgr2=0x7fb2ec0cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec0cbbe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009910 tx=0x7fb2e4009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- 172.21.15.22:0/3052213570 shutdown_connections 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec0cbbe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/3052213570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec159c30 0x7fb2ec0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- 172.21.15.22:0/3052213570 >> 172.21.15.22:0/3052213570 conn(0x7fb2ec06a5f0 msgr2=0x7fb2ec0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- 172.21.15.22:0/3052213570 shutdown_connections 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.339+0000 7fb2f18f6700 1 -- 172.21.15.22:0/3052213570 wait complete. 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 Processor -- start 2023-06-23T18:04:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 -- start start 2023-06-23T18:04:51.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0affd0 0x7fb2ec0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec159c30 0x7fb2ec099c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0c9670 con 0x7fb2ec071450 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b1e40 con 0x7fb2ec159c30 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f18f6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b2120 con 0x7fb2ec0affd0 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42060/0 (socket says 172.21.15.22:42060) 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 -- 172.21.15.22:0/1982256320 learned_addr learned my addr 172.21.15.22:0/1982256320 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f10f5700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec159c30 0x7fb2ec099c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2ebfff700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0affd0 0x7fb2ec0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 -- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0affd0 msgr2=0x7fb2ec0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0affd0 0x7fb2ec0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 -- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec159c30 msgr2=0x7fb2ec099c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec159c30 0x7fb2ec099c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 -- 172.21.15.22:0/1982256320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4009570 con 0x7fb2ec071450 2023-06-23T18:04:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.343+0000 7fb2f08f4700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e840 tx=0x7fb2e000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00095a0 con 0x7fb2ec071450 2023-06-23T18:04:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0013680 con 0x7fb2ec071450 2023-06-23T18:04:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec0b2400 con 0x7fb2ec071450 2023-06-23T18:04:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00139f0 con 0x7fb2ec071450 2023-06-23T18:04:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec153360 con 0x7fb2ec071450 2023-06-23T18:04:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec04e4c0 con 0x7fb2ec071450 2023-06-23T18:04:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb2e0013b50 con 0x7fb2ec071450 2023-06-23T18:04:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 0x7fb2d80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb2e0083ef0 con 0x7fb2ec071450 2023-06-23T18:04:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.347+0000 7fb2ebfff700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 0x7fb2d80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.351+0000 7fb2ebfff700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 0x7fb2d80687a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b3f60 tx=0x7fb2dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.355+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb2e0054a50 con 0x7fb2ec071450 2023-06-23T18:04:51.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.551+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2ec0cc370 con 0x7fb2ec071450 2023-06-23T18:04:51.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.551+0000 7fb2e97fa700 1 -- 172.21.15.22:0/1982256320 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fb2e00545e0 con 0x7fb2ec071450 2023-06-23T18:04:51.563 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:51.563 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:51.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 msgr2=0x7fb2d80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 0x7fb2d80687a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b3f60 tx=0x7fb2dc009040 comp rx=0 tx=0).stop 2023-06-23T18:04:51.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 msgr2=0x7fb2ec152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e840 tx=0x7fb2e000eb50 comp rx=0 tx=0).stop 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 shutdown_connections 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb2d80662f0 0x7fb2d80687a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2ec159c30 0x7fb2ec099c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2ec0affd0 0x7fb2ec0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 --2- 172.21.15.22:0/1982256320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ec071450 0x7fb2ec152e20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 >> 172.21.15.22:0/1982256320 conn(0x7fb2ec06a5f0 msgr2=0x7fb2ec0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 shutdown_connections 2023-06-23T18:04:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.555+0000 7fb2f18f6700 1 -- 172.21.15.22:0/1982256320 wait complete. 2023-06-23T18:04:51.567 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:51.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:04:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.683+0000 7f39e5631700 1 Processor -- start 2023-06-23T18:04:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.683+0000 7f39e5631700 1 -- start start 2023-06-23T18:04:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f39e00c9c60 0x7f39e00ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00cdaf0 con 0x7f39e0157f00 2023-06-23T18:04:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00cca70 con 0x7f39e00c9c60 2023-06-23T18:04:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c9780 con 0x7f39e00cd160 2023-06-23T18:04:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39dffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f39e0157f00 0x7f39e00cc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42724/0 (socket says 172.21.15.22:42724) 2023-06-23T18:04:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39dffff700 1 -- 172.21.15.22:0/1267251306 learned_addr learned my addr 172.21.15.22:0/1267251306 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39d77fe700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 --2- 172.21.15.22:0/1267251306 >> v2:172.21.15.169:3300/0 conn(0x7f39e00c9c60 0x7f39e00ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 -- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 msgr2=0x7f39e00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39df7fe700 1 -- 172.21.15.22:0/1267251306 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953430286 0 0) 0x7f39d0001110 con 0x7f39e0157f00 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 -- 172.21.15.22:0/1267251306 >> v1:172.21.15.22:6789/0 conn(0x7f39e0157f00 legacy=0x7f39e00cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 -- 172.21.15.22:0/1267251306 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00c4c50 con 0x7f39e00c9c60 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e4e30700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e00c9c60 0x7f39e00ca0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39d800aed0 tx=0x7f39d800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30e4f6e94cc4ce4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39df7fe700 1 -- 172.21.15.22:0/1267251306 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800f070 con 0x7f39e00c9c60 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39df7fe700 1 -- 172.21.15.22:0/1267251306 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d800a2e0 con 0x7f39e00c9c60 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39df7fe700 1 -- 172.21.15.22:0/1267251306 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800cbd0 con 0x7f39e00c9c60 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e00c9c60 msgr2=0x7f39e00ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e00c9c60 0x7f39e00ca0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39d800aed0 tx=0x7f39d800c330 comp rx=0 tx=0).stop 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- 172.21.15.22:0/1267251306 shutdown_connections 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e00c9c60 0x7f39e00ca0e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 --2- 172.21.15.22:0/1267251306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- 172.21.15.22:0/1267251306 >> 172.21.15.22:0/1267251306 conn(0x7f39e00cb2a0 msgr2=0x7f39e00cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- 172.21.15.22:0/1267251306 shutdown_connections 2023-06-23T18:04:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.687+0000 7f39e5631700 1 -- 172.21.15.22:0/1267251306 wait complete. 2023-06-23T18:04:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 Processor -- start 2023-06-23T18:04:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 -- start start 2023-06-23T18:04:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e0157f00 0x7f39e00c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0150a10 con 0x7f39e00c9c60 2023-06-23T18:04:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b4590 con 0x7f39e0157f00 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39e5631700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b2b10 con 0x7f39e00cd160 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.691+0000 7f39dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42070/0 (socket says 172.21.15.22:42070) 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 -- 172.21.15.22:0/1704526778 learned_addr learned my addr 172.21.15.22:0/1704526778 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39d77fe700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39e4e30700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e0157f00 0x7f39e00c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 -- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 msgr2=0x7f39e00b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 -- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e0157f00 msgr2=0x7f39e00c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e0157f00 0x7f39e00c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 -- 172.21.15.22:0/1704526778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d800ab80 con 0x7f39e00c9c60 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dffff700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c750 tx=0x7f39d000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004ca0 con 0x7f39e00c9c60 2023-06-23T18:04:51.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0004e00 con 0x7f39e00c9c60 2023-06-23T18:04:51.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e00b2df0 con 0x7f39e00c9c60 2023-06-23T18:04:51.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0007310 con 0x7f39e00c9c60 2023-06-23T18:04:51.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e00b33b0 con 0x7f39e00c9c60 2023-06-23T18:04:51.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e004e4c0 con 0x7f39e00c9c60 2023-06-23T18:04:51.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f39d0007470 con 0x7f39e00c9c60 2023-06-23T18:04:51.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.695+0000 7f39dd7fa700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 0x7f39c00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:51.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.699+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f39d0083bf0 con 0x7f39e00c9c60 2023-06-23T18:04:51.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.699+0000 7f39d77fe700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 0x7f39c00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:51.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.699+0000 7f39d77fe700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 0x7f39c00687a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f39e006b050 tx=0x7f39c8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:51.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.703+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f39d0050d40 con 0x7f39e00c9c60 2023-06-23T18:04:51.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.899+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e00cdaf0 con 0x7f39e00c9c60 2023-06-23T18:04:51.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39dd7fa700 1 -- 172.21.15.22:0/1704526778 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f39e00cdaf0 con 0x7f39e00c9c60 2023-06-23T18:04:51.914 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:51.915 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3859658146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3859658146},{"type":"v1","addr":"172.21.15.169:6833","nonce":3859658146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/957334219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":957334219},{"type":"v1","addr":"172.21.15.169:6835","nonce":957334219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6836/1477666510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1477666510},{"type":"v1","addr":"172.21.15.22:6836","nonce":1477666510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:03:37.725874+0000","modified":"2023-06-23T18:04:00.692712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6837/2486746630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":2486746630},{"type":"v1","addr":"172.21.15.22:6837","nonce":2486746630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:04:51.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 msgr2=0x7f39c00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 0x7f39c00687a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f39e006b050 tx=0x7f39c8009240 comp rx=0 tx=0).stop 2023-06-23T18:04:51.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 msgr2=0x7f39e01504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:51.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c750 tx=0x7f39d000ca60 comp rx=0 tx=0).stop 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 shutdown_connections 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f39c00662f0 0x7f39c00687a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39e0157f00 0x7f39e00c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39e00cd160 0x7f39e00b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 --2- 172.21.15.22:0/1704526778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39e00c9c60 0x7f39e01504d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 >> 172.21.15.22:0/1704526778 conn(0x7f39e00cb2a0 msgr2=0x7f39e0151260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:51.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 shutdown_connections 2023-06-23T18:04:51.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:51.903+0000 7f39e5631700 1 -- 172.21.15.22:0/1704526778 wait complete. 2023-06-23T18:04:51.918 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-06-23T18:04:51.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:04:52.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 Processor -- start 2023-06-23T18:04:52.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- start start 2023-06-23T18:04:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa15415fe20 0x7fa154164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa1541613b0 0x7fa1541601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1540ce280 0x7fa154070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa154161800 con 0x7fa15415fe20 2023-06-23T18:04:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa154160730 con 0x7fa1541613b0 2023-06-23T18:04:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154160f20 con 0x7fa1540ce280 2023-06-23T18:04:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa15415fe20 0x7fa154164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa15415fe20 0x7fa154164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42072/0 (socket says 172.21.15.22:42072) 2023-06-23T18:04:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 -- 172.21.15.22:0/266372168 learned_addr learned my addr 172.21.15.22:0/266372168 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa152ffd700 1 --2- 172.21.15.22:0/266372168 >> v2:172.21.15.169:3300/0 conn(0x7fa1541613b0 0x7fa1541601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa153fff700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1540ce280 0x7fa154070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 -- 172.21.15.22:0/266372168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1540ce280 msgr2=0x7fa154070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1540ce280 0x7fa154070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 -- 172.21.15.22:0/266372168 >> v2:172.21.15.169:3300/0 conn(0x7fa1541613b0 msgr2=0x7fa1541601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:52.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 --2- 172.21.15.22:0/266372168 >> v2:172.21.15.169:3300/0 conn(0x7fa1541613b0 0x7fa1541601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 -- 172.21.15.22:0/266372168 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa154160940 con 0x7fa15415fe20 2023-06-23T18:04:52.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1537fe700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa15415fe20 0x7fa154164920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa154070540 tx=0x7fa13c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb70baeade658db server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1527fc700 1 -- 172.21.15.22:0/266372168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c011830 con 0x7fa15415fe20 2023-06-23T18:04:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1527fc700 1 -- 172.21.15.22:0/266372168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c011990 con 0x7fa15415fe20 2023-06-23T18:04:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa1527fc700 1 -- 172.21.15.22:0/266372168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c011d20 con 0x7fa15415fe20 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- 172.21.15.22:0/266372168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa15415fe20 msgr2=0x7fa154164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa15415fe20 0x7fa154164920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa154070540 tx=0x7fa13c009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- 172.21.15.22:0/266372168 shutdown_connections 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1540ce280 0x7fa154070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- 172.21.15.22:0/266372168 >> v2:172.21.15.169:3300/0 conn(0x7fa1541613b0 0x7fa1541601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 --2- 172.21.15.22:0/266372168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa15415fe20 0x7fa154164920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- 172.21.15.22:0/266372168 >> 172.21.15.22:0/266372168 conn(0x7fa15406aa70 msgr2=0x7fa15406ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:52.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- 172.21.15.22:0/266372168 shutdown_connections 2023-06-23T18:04:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.035+0000 7fa158e9e700 1 -- 172.21.15.22:0/266372168 wait complete. 2023-06-23T18:04:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 Processor -- start 2023-06-23T18:04:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 -- start start 2023-06-23T18:04:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa15415fe20 0x7fa1541f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 0x7fa1541623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541f9d50 con 0x7fa1540ce280 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540c8680 con 0x7fa1541613b0 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa158e9e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540c8960 con 0x7fa15415fe20 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42076/0 (socket says 172.21.15.22:42076) 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 -- 172.21.15.22:0/530331856 learned_addr learned my addr 172.21.15.22:0/530331856 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa153fff700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 0x7fa1541623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa152ffd700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa15415fe20 0x7fa1541f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 -- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa15415fe20 msgr2=0x7fa1541f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa15415fe20 0x7fa1541f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 -- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 msgr2=0x7fa1541623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.039+0000 7fa1537fe700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 0x7fa1541623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa1537fe700 1 -- 172.21.15.22:0/530331856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009570 con 0x7fa1540ce280 2023-06-23T18:04:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa153fff700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 0x7fa1541623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa1537fe700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa154070340 tx=0x7fa13c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004240 con 0x7fa1540ce280 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c01f070 con 0x7fa1540ce280 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540c8c40 con 0x7fa1540ce280 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004600 con 0x7fa1540ce280 2023-06-23T18:04:52.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1540c9200 con 0x7fa1540ce280 2023-06-23T18:04:52.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15404e4c0 con 0x7fa1540ce280 2023-06-23T18:04:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.043+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa13c021030 con 0x7fa1540ce280 2023-06-23T18:04:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.047+0000 7fa14bfff700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 0x7fa1340687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.047+0000 7fa152ffd700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 0x7fa1340687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.047+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa13c085040 con 0x7fa1540ce280 2023-06-23T18:04:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.047+0000 7fa152ffd700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 0x7fa1340687d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa15415dac0 tx=0x7fa14c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.051+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa13c052340 con 0x7fa1540ce280 2023-06-23T18:04:52.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.215+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1540b50e0 con 0x7fa1540ce280 2023-06-23T18:04:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa14bfff700 1 -- 172.21.15.22:0/530331856 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20204 (secure 0 0 0) 0x7fa13c015020 con 0x7fa1540ce280 2023-06-23T18:04:52.234 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:52.235 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:27.422030+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:03:36.275123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T18:03:36.707871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-23T18:03:39.789766+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:04:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 msgr2=0x7fa1340687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 0x7fa1340687d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa15415dac0 tx=0x7fa14c00b040 comp rx=0 tx=0).stop 2023-06-23T18:04:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 msgr2=0x7fa15414fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa154070340 tx=0x7fa13c004770 comp rx=0 tx=0).stop 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 shutdown_connections 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa134066320 0x7fa1340687d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1541613b0 0x7fa1541623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa15415fe20 0x7fa1541f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 --2- 172.21.15.22:0/530331856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1540ce280 0x7fa15414fe20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 >> 172.21.15.22:0/530331856 conn(0x7fa15406aa70 msgr2=0x7fa15415e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 shutdown_connections 2023-06-23T18:04:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.219+0000 7fa158e9e700 1 -- 172.21.15.22:0/530331856 wait complete. 2023-06-23T18:04:52.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-23T18:04:52.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.355+0000 7efd188f5700 1 Processor -- start 2023-06-23T18:04:52.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.355+0000 7efd188f5700 1 -- start start 2023-06-23T18:04:52.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd140cf480 0x7efd140ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd140d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd140b7ec0 con 0x7efd140b7af0 2023-06-23T18:04:52.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd140b1ad0 con 0x7efd140cf480 2023-06-23T18:04:52.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd140d0a10 con 0x7efd140cc9e0 2023-06-23T18:04:52.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd140cf480 0x7efd140ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd140cf480 0x7efd140ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42080/0 (socket says 172.21.15.22:42080) 2023-06-23T18:04:52.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 -- 172.21.15.22:0/2252621920 learned_addr learned my addr 172.21.15.22:0/2252621920 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd13fff700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd140d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 -- 172.21.15.22:0/2252621920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 msgr2=0x7efd140d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:52.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd140d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 -- 172.21.15.22:0/2252621920 >> v1:172.21.15.169:6789/0 conn(0x7efd140b7af0 legacy=0x7efd140ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 -- 172.21.15.22:0/2252621920 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140d0cb0 con 0x7efd140cf480 2023-06-23T18:04:52.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd0bfff700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd140ce540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009910 tx=0x7efcfc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5643b50996eabe server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd12ffd700 1 -- 172.21.15.22:0/2252621920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011830 con 0x7efd140cf480 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd12ffd700 1 -- 172.21.15.22:0/2252621920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc011990 con 0x7efd140cf480 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd12ffd700 1 -- 172.21.15.22:0/2252621920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011d40 con 0x7efd140cf480 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- 172.21.15.22:0/2252621920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 msgr2=0x7efd140ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd140ce540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009910 tx=0x7efcfc009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- 172.21.15.22:0/2252621920 shutdown_connections 2023-06-23T18:04:52.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd140d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 --2- 172.21.15.22:0/2252621920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd140ce540 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- 172.21.15.22:0/2252621920 >> 172.21.15.22:0/2252621920 conn(0x7efd14069960 msgr2=0x7efd14069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:52.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- 172.21.15.22:0/2252621920 shutdown_connections 2023-06-23T18:04:52.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.359+0000 7efd188f5700 1 -- 172.21.15.22:0/2252621920 wait complete. 2023-06-23T18:04:52.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 Processor -- start 2023-06-23T18:04:52.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- start start 2023-06-23T18:04:52.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd140b7af0 0x7efd140c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd1415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd140c7070 con 0x7efd140cf480 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1415d900 con 0x7efd140b7af0 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd140c6480 con 0x7efd140cc9e0 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42094/0 (socket says 172.21.15.22:42094) 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 -- 172.21.15.22:0/2863746374 learned_addr learned my addr 172.21.15.22:0/2863746374 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd0bfff700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd1415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd137fe700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd140b7af0 0x7efd140c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 -- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 msgr2=0x7efd1415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd1415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 -- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd140b7af0 msgr2=0x7efd140c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd140b7af0 0x7efd140c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 -- 172.21.15.22:0/2863746374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc009570 con 0x7efd140cf480 2023-06-23T18:04:52.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd13fff700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009dd0 tx=0x7efd0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c009c40 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c00f040 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00ceb0 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd140c6ab0 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.363+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd140c6760 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.367+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1404e4c0 con 0x7efd140cf480 2023-06-23T18:04:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.367+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7efd0c014480 con 0x7efd140cf480 2023-06-23T18:04:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.367+0000 7efd10ff9700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 0x7efcf40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.367+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd0c084ff0 con 0x7efd140cf480 2023-06-23T18:04:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.367+0000 7efd137fe700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 0x7efcf40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.371+0000 7efd137fe700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 0x7efcf40687a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd04000f60 tx=0x7efd04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.375+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7efd0c055be0 con 0x7efd140cf480 2023-06-23T18:04:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.571+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efd140b4740 con 0x7efd140cf480 2023-06-23T18:04:52.600 INFO:tasks.ceph.mon.a.smithi022.stderr:2023-06-23T18:04:52.587+0000 7f9c130e3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-23T18:04:52.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.599+0000 7efd10ff9700 1 -- 172.21.15.22:0/2863746374 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd0c018020 con 0x7efd140cf480 2023-06-23T18:04:52.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 msgr2=0x7efcf40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 0x7efcf40687a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd04000f60 tx=0x7efd04009040 comp rx=0 tx=0).stop 2023-06-23T18:04:52.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 msgr2=0x7efd141557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009dd0 tx=0x7efd0c00c3a0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 shutdown_connections 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd140cf480 0x7efd141557d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd140cc9e0 0x7efd1415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7efcf40662f0 0x7efcf40687a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 --2- 172.21.15.22:0/2863746374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd140b7af0 0x7efd140c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 >> 172.21.15.22:0/2863746374 conn(0x7efd14069960 msgr2=0x7efd140b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 shutdown_connections 2023-06-23T18:04:52.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.603+0000 7efd188f5700 1 -- 172.21.15.22:0/2863746374 wait complete. 2023-06-23T18:04:52.617 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-23T18:04:52.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-23T18:04:52.652 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2023-06-23T18:04:52.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 Processor -- start 2023-06-23T18:04:52.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 -- start start 2023-06-23T18:04:52.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5140b1920 0x7fc5140cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5140d21d0 0x7fc5140d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140d1d40 con 0x7fc5140cca20 2023-06-23T18:04:52.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140b1cf0 con 0x7fc5140b1920 2023-06-23T18:04:52.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51ae32700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140cf130 con 0x7fc5140d21d0 2023-06-23T18:04:52.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5140b1920 0x7fc5140cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5140b1920 0x7fc5140cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42128/0 (socket says 172.21.15.22:42128) 2023-06-23T18:04:52.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 -- 172.21.15.22:0/1477474308 learned_addr learned my addr 172.21.15.22:0/1477474308 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc51a631700 1 --2- 172.21.15.22:0/1477474308 >> v2:172.21.15.169:3300/0 conn(0x7fc5140d21d0 0x7fc5140d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 -- 172.21.15.22:0/1477474308 >> v2:172.21.15.169:3300/0 conn(0x7fc5140d21d0 msgr2=0x7fc5140d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:52.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 --2- 172.21.15.22:0/1477474308 >> v2:172.21.15.169:3300/0 conn(0x7fc5140d21d0 0x7fc5140d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 -- 172.21.15.22:0/1477474308 >> v1:172.21.15.169:6789/0 conn(0x7fc5140cca20 legacy=0x7fc5140d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 -- 172.21.15.22:0/1477474308 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140cf410 con 0x7fc5140b1920 2023-06-23T18:04:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc519e30700 1 --2- 172.21.15.22:0/1477474308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140b1920 0x7fc5140cffa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc5140d1140 tx=0x7fc510009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c1c23ffa6aa88f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.739+0000 7fc518e2e700 1 -- 172.21.15.22:0/1477474308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510004230 con 0x7fc5140b1920 2023-06-23T18:04:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc518e2e700 1 -- 172.21.15.22:0/1477474308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510004390 con 0x7fc5140b1920 2023-06-23T18:04:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc518e2e700 1 -- 172.21.15.22:0/1477474308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100036c0 con 0x7fc5140b1920 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- 172.21.15.22:0/1477474308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140b1920 msgr2=0x7fc5140cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- 172.21.15.22:0/1477474308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140b1920 0x7fc5140cffa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc5140d1140 tx=0x7fc510009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- 172.21.15.22:0/1477474308 shutdown_connections 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- 172.21.15.22:0/1477474308 >> v2:172.21.15.169:3300/0 conn(0x7fc5140d21d0 0x7fc5140d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- 172.21.15.22:0/1477474308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140b1920 0x7fc5140cffa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- 172.21.15.22:0/1477474308 >> 172.21.15.22:0/1477474308 conn(0x7fc514069960 msgr2=0x7fc514069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:52.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- 172.21.15.22:0/1477474308 shutdown_connections 2023-06-23T18:04:52.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- 172.21.15.22:0/1477474308 wait complete. 2023-06-23T18:04:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 Processor -- start 2023-06-23T18:04:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- start start 2023-06-23T18:04:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5140b1920 0x7fc514153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5140cca20 0x7fc5140cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140b6140 con 0x7fc5140d21d0 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140b5530 con 0x7fc5140b1920 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51ae32700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140b5810 con 0x7fc5140cca20 2023-06-23T18:04:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51a631700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51a631700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42130/0 (socket says 172.21.15.22:42130) 2023-06-23T18:04:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51a631700 1 -- 172.21.15.22:0/2618996369 learned_addr learned my addr 172.21.15.22:0/2618996369 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc51962f700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5140cca20 0x7fc5140cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.743+0000 7fc519e30700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5140b1920 0x7fc514153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 -- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5140cca20 msgr2=0x7fc5140cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5140cca20 0x7fc5140cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 -- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5140b1920 msgr2=0x7fc514153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5140b1920 0x7fc514153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 -- 172.21.15.22:0/2618996369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510009570 con 0x7fc5140d21d0 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51a631700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007e60 tx=0x7fc50c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00ed30 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5140b5af0 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c014420 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0147d0 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5141538c0 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.747+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514154c40 con 0x7fc5140d21d0 2023-06-23T18:04:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.751+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fc50c014930 con 0x7fc5140d21d0 2023-06-23T18:04:52.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.751+0000 7fc50a7fc700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 0x7fc4f80687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:52.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.751+0000 7fc519e30700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 0x7fc4f80687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:52.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.751+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc50c0847d0 con 0x7fc5140d21d0 2023-06-23T18:04:52.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.751+0000 7fc519e30700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 0x7fc4f80687a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5140d1140 tx=0x7fc510008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:52.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.755+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fc50c0517b0 con 0x7fc5140d21d0 2023-06-23T18:04:52.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:52.959+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc5140cf6c0 con 0x7fc5140d21d0 2023-06-23T18:04:53.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.603+0000 7fc50a7fc700 1 -- 172.21.15.22:0/2618996369 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc50c054dd0 con 0x7fc5140d21d0 2023-06-23T18:04:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.607+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 msgr2=0x7fc4f80687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.607+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 0x7fc4f80687a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5140d1140 tx=0x7fc510008030 comp rx=0 tx=0).stop 2023-06-23T18:04:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.607+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 msgr2=0x7fc51415a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.607+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007e60 tx=0x7fc50c00e3c0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 shutdown_connections 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5140d21d0 0x7fc51415a7a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5140cca20 0x7fc5140cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fc4f80662f0 0x7fc4f80687a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 --2- 172.21.15.22:0/2618996369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5140b1920 0x7fc514153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 >> 172.21.15.22:0/2618996369 conn(0x7fc514069960 msgr2=0x7fc5140d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 shutdown_connections 2023-06-23T18:04:53.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.611+0000 7fc51ae32700 1 -- 172.21.15.22:0/2618996369 wait complete. 2023-06-23T18:04:53.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-23T18:04:53.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 Processor -- start 2023-06-23T18:04:53.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 -- start start 2023-06-23T18:04:53.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 0x7f82240cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240ccb30 con 0x7f82240d0730 2023-06-23T18:04:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240cfcb0 con 0x7f82240d1f30 2023-06-23T18:04:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82240d2880 con 0x7f82240cf3a0 2023-06-23T18:04:53.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8223fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f82240d0730 0x7f82240ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42738/0 (socket says 172.21.15.22:42738) 2023-06-23T18:04:53.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8223fff700 1 -- 172.21.15.22:0/452604483 learned_addr learned my addr 172.21.15.22:0/452604483 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:53.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82237fe700 1 --2- 172.21.15.22:0/452604483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 0x7f82240cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:53.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961371474 0 0) 0x7f82240ccb30 con 0x7f82240d0730 2023-06-23T18:04:53.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82100035d0 con 0x7f82240d0730 2023-06-23T18:04:53.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376375850 0 0) 0x7f82240cfcb0 con 0x7f82240d1f30 2023-06-23T18:04:53.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82240ccb30 con 0x7f82240d1f30 2023-06-23T18:04:53.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82237fe700 1 -- 172.21.15.22:0/452604483 >> v1:172.21.15.169:6789/0 conn(0x7f82240d1f30 legacy=0x7f82240d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82237fe700 1 -- 172.21.15.22:0/452604483 >> v1:172.21.15.22:6789/0 conn(0x7f82240d0730 legacy=0x7f82240ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82237fe700 1 -- 172.21.15.22:0/452604483 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240d2b60 con 0x7f82240cf3a0 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82237fe700 1 --2- 172.21.15.22:0/452604483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 0x7f82240cf770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8214009e10 tx=0x7f821400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12228763e943a329 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e070 con 0x7f82240cf3a0 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400b910 con 0x7f82240cf3a0 2023-06-23T18:04:53.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f82227fc700 1 -- 172.21.15.22:0/452604483 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400bcc0 con 0x7f82240cf3a0 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 -- 172.21.15.22:0/452604483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 msgr2=0x7f82240cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.799+0000 7f8228c4c700 1 --2- 172.21.15.22:0/452604483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 0x7f82240cf770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8214009e10 tx=0x7f821400b330 comp rx=0 tx=0).stop 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/452604483 shutdown_connections 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 --2- 172.21.15.22:0/452604483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240cf3a0 0x7f82240cf770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/452604483 >> 172.21.15.22:0/452604483 conn(0x7f822406a280 msgr2=0x7f822409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/452604483 shutdown_connections 2023-06-23T18:04:53.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/452604483 wait complete. 2023-06-23T18:04:53.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 Processor -- start 2023-06-23T18:04:53.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- start start 2023-06-23T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240d0730 0x7f822415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82240d1f30 0x7f822406c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:53.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82241f5290 con 0x7f82240cf3a0 2023-06-23T18:04:53.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822415c630 con 0x7f82240d1f30 2023-06-23T18:04:53.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82241f4960 con 0x7f82240d0730 2023-06-23T18:04:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42134/0 (socket says 172.21.15.22:42134) 2023-06-23T18:04:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 -- 172.21.15.22:0/818754449 learned_addr learned my addr 172.21.15.22:0/818754449 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8222ffd700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240d0730 0x7f822415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8223fff700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82240d1f30 0x7f822406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 -- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240d0730 msgr2=0x7f822415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240d0730 0x7f822415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 -- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82240d1f30 msgr2=0x7f822406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82240d1f30 0x7f822406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 -- 172.21.15.22:0/818754449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214009a70 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f82237fe700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8214000f80 tx=0x7f8214004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140098f0 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214020070 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82241f4c40 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140276a0 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.803+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82241f5720 con 0x7f82240cf3a0 2023-06-23T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822404e4c0 con 0x7f82240cf3a0 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f8214027800 con 0x7f82240cf3a0 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f821bfff700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 0x7f82040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f8214022070 con 0x7f82240cf3a0 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f8222ffd700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 0x7f82040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.807+0000 7f8222ffd700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 0x7f82040687a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82240cc4c0 tx=0x7f820c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.811+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f8214052f20 con 0x7f82240cf3a0 2023-06-23T18:04:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:53.975+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f82240b18a0 con 0x7f82240cf3a0 2023-06-23T18:04:54.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.627+0000 7f821bfff700 1 -- 172.21.15.22:0/818754449 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8214056540 con 0x7f82240cf3a0 2023-06-23T18:04:54.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 msgr2=0x7f82040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 0x7f82040687a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82240cc4c0 tx=0x7f820c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:54.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 msgr2=0x7f822415eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8214000f80 tx=0x7f8214004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 shutdown_connections 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f82040662f0 0x7f82040687a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82240d1f30 0x7f822406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82240d0730 0x7f822415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.635+0000 7f8228c4c700 1 --2- 172.21.15.22:0/818754449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82240cf3a0 0x7f822415eae0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.639+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 >> 172.21.15.22:0/818754449 conn(0x7f822406a280 msgr2=0x7f82240b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:54.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.639+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 shutdown_connections 2023-06-23T18:04:54.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.639+0000 7f8228c4c700 1 -- 172.21.15.22:0/818754449 wait complete. 2023-06-23T18:04:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2023-06-23T18:04:54.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-23T18:04:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 Processor -- start 2023-06-23T18:04:54.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 -- start start 2023-06-23T18:04:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f6c0cca10 0x7f9f6c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6c160900 con 0x7f9f6c0b1400 2023-06-23T18:04:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6c0d1730 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f71342700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6c0cd060 con 0x7f9f6c0cca10 2023-06-23T18:04:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.779+0000 7f9f6bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9f6c0b1400 0x7f9f6c0d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42740/0 (socket says 172.21.15.22:42740) 2023-06-23T18:04:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6bfff700 1 -- 172.21.15.22:0/1697720702 learned_addr learned my addr 172.21.15.22:0/1697720702 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f637fe700 1 --2- 172.21.15.22:0/1697720702 >> v2:172.21.15.169:3300/0 conn(0x7f9f6c0cca10 0x7f9f6c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856368477 0 0) 0x7f9f6c0d1730 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f4c0035d0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162239634 0 0) 0x7f9f54001110 con 0x7f9f6c0b1400 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f6c0d1730 con 0x7f9f6c0b1400 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f637fe700 1 -- 172.21.15.22:0/1697720702 >> v1:172.21.15.169:6789/0 conn(0x7f9f6c0b3ea0 legacy=0x7f9f6c0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f637fe700 1 -- 172.21.15.22:0/1697720702 >> v1:172.21.15.22:6789/0 conn(0x7f9f6c0b1400 legacy=0x7f9f6c0d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f637fe700 1 -- 172.21.15.22:0/1697720702 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0cd340 con 0x7f9f6c0cca10 2023-06-23T18:04:54.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f637fe700 1 --2- 172.21.15.22:0/1697720702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c1603c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009850 tx=0x7f9f5c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dad4cd08268aeed server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011830 con 0x7f9f6c0cca10 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c011990 con 0x7f9f6c0cca10 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f6b7fe700 1 -- 172.21.15.22:0/1697720702 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011d40 con 0x7f9f6c0cca10 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 -- 172.21.15.22:0/1697720702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 msgr2=0x7f9f6c1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 --2- 172.21.15.22:0/1697720702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c1603c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009850 tx=0x7f9f5c009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 -- 172.21.15.22:0/1697720702 shutdown_connections 2023-06-23T18:04:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 --2- 172.21.15.22:0/1697720702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c1603c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 -- 172.21.15.22:0/1697720702 >> 172.21.15.22:0/1697720702 conn(0x7f9f6c069960 msgr2=0x7f9f6c069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 -- 172.21.15.22:0/1697720702 shutdown_connections 2023-06-23T18:04:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.783+0000 7f9f71342700 1 -- 172.21.15.22:0/1697720702 wait complete. 2023-06-23T18:04:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 Processor -- start 2023-06-23T18:04:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- start start 2023-06-23T18:04:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f6c0b1400 0x7f9f6c0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:54.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:54.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0cade0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c155de0 con 0x7f9f6c0cca10 2023-06-23T18:04:54.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0c9800 con 0x7f9f6c0b1400 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42150/0 (socket says 172.21.15.22:42150) 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 -- 172.21.15.22:0/1546680966 learned_addr learned my addr 172.21.15.22:0/1546680966 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f6bfff700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f6c0b1400 0x7f9f6c0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f70b41700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:54.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 -- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f6c0b1400 msgr2=0x7f9f6c0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f6c0b1400 0x7f9f6c0ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 -- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 msgr2=0x7f9f6c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:54.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:54.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 -- 172.21.15.22:0/1546680966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0094b0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f637fe700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0090b0 tx=0x7f9f5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c019ab0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c01e070 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6c0c9ae0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c019e70 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.787+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c0ca0a0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.791+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6c04e4c0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.791+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f9f5c02d450 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.791+0000 7f9f697fa700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 0x7f9f500687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:54.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.791+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f9f5c084f10 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.791+0000 7f9f6bfff700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 0x7f9f500687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:54.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.795+0000 7f9f6bfff700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 0x7f9f500687d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000f60 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:54.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.799+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f9f5c0521b0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:54.963+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f6c0d2bc0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:55.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.631+0000 7f9f697fa700 1 -- 172.21.15.22:0/1546680966 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9f5c0150d0 con 0x7f9f6c0b3ea0 2023-06-23T18:04:55.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 msgr2=0x7f9f500687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 0x7f9f500687d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000f60 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 msgr2=0x7f9f6c15e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0090b0 tx=0x7f9f5c004770 comp rx=0 tx=0).stop 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 shutdown_connections 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f9f50066320 0x7f9f500687d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f6c0cca10 0x7f9f6c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f6c0b3ea0 0x7f9f6c15e560 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 --2- 172.21.15.22:0/1546680966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f6c0b1400 0x7f9f6c0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 >> 172.21.15.22:0/1546680966 conn(0x7f9f6c069960 msgr2=0x7f9f6c0b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 shutdown_connections 2023-06-23T18:04:55.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.635+0000 7f9f71342700 1 -- 172.21.15.22:0/1546680966 wait complete. 2023-06-23T18:04:55.651 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2023-06-23T18:04:55.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-06-23T18:04:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 Processor -- start 2023-06-23T18:04:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 -- start start 2023-06-23T18:04:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 0x7fd2180ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd2180cf1b0 0x7fd21815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180ce780 0x7fd21815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180b62e0 con 0x7fd2180cf1b0 2023-06-23T18:04:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21815a080 con 0x7fd2180cde70 2023-06-23T18:04:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21de2d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180cc830 con 0x7fd2180ce780 2023-06-23T18:04:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd2180cf1b0 0x7fd21815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd2180cf1b0 0x7fd21815d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42152/0 (socket says 172.21.15.22:42152) 2023-06-23T18:04:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 -- 172.21.15.22:0/2596340172 learned_addr learned my addr 172.21.15.22:0/2596340172 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21ce2b700 1 --2- 172.21.15.22:0/2596340172 >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 0x7fd2180ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21d62c700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180ce780 0x7fd21815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 -- 172.21.15.22:0/2596340172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180ce780 msgr2=0x7fd21815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180ce780 0x7fd21815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 -- 172.21.15.22:0/2596340172 >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 msgr2=0x7fd2180ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 --2- 172.21.15.22:0/2596340172 >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 0x7fd2180ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 -- 172.21.15.22:0/2596340172 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180cd3b0 con 0x7fd2180cf1b0 2023-06-23T18:04:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd21ce2b700 1 --2- 172.21.15.22:0/2596340172 >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 0x7fd2180ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.779+0000 7fd217fff700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180cf1b0 0x7fd21815d7a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd21806eef0 tx=0x7fd204009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616d65db251dc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd2177fe700 1 -- 172.21.15.22:0/2596340172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011830 con 0x7fd2180cf1b0 2023-06-23T18:04:55.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd2177fe700 1 -- 172.21.15.22:0/2596340172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204011990 con 0x7fd2180cf1b0 2023-06-23T18:04:55.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd2177fe700 1 -- 172.21.15.22:0/2596340172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011d20 con 0x7fd2180cf1b0 2023-06-23T18:04:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 -- 172.21.15.22:0/2596340172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180cf1b0 msgr2=0x7fd21815d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180cf1b0 0x7fd21815d7a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd21806eef0 tx=0x7fd204009bf0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 -- 172.21.15.22:0/2596340172 shutdown_connections 2023-06-23T18:04:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180ce780 0x7fd21815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 --2- 172.21.15.22:0/2596340172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180cf1b0 0x7fd21815d7a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 --2- 172.21.15.22:0/2596340172 >> v2:172.21.15.169:3300/0 conn(0x7fd2180cde70 0x7fd2180ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 -- 172.21.15.22:0/2596340172 >> 172.21.15.22:0/2596340172 conn(0x7fd218069960 msgr2=0x7fd218069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:55.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 -- 172.21.15.22:0/2596340172 shutdown_connections 2023-06-23T18:04:55.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.783+0000 7fd21de2d700 1 -- 172.21.15.22:0/2596340172 wait complete. 2023-06-23T18:04:55.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 Processor -- start 2023-06-23T18:04:55.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- start start 2023-06-23T18:04:55.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180cde70 0x7fd2180c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180cf1b0 0x7fd218144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180c65e0 con 0x7fd2180ce780 2023-06-23T18:04:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181549c0 con 0x7fd2180cf1b0 2023-06-23T18:04:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd218154ca0 con 0x7fd2180cde70 2023-06-23T18:04:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42164/0 (socket says 172.21.15.22:42164) 2023-06-23T18:04:55.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 -- 172.21.15.22:0/1476625476 learned_addr learned my addr 172.21.15.22:0/1476625476 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21ce2b700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180cde70 0x7fd2180c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21d62c700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180cf1b0 0x7fd218144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 -- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180cde70 msgr2=0x7fd2180c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:55.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180cde70 0x7fd2180c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 -- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180cf1b0 msgr2=0x7fd218144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:55.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180cf1b0 0x7fd218144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:55.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 -- 172.21.15.22:0/1476625476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009570 con 0x7fd2180ce780 2023-06-23T18:04:55.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd217fff700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd204009860 tx=0x7fd204004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:55.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204004240 con 0x7fd2180ce780 2023-06-23T18:04:55.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20401f070 con 0x7fd2180ce780 2023-06-23T18:04:55.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd218154f80 con 0x7fd2180ce780 2023-06-23T18:04:55.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040045c0 con 0x7fd2180ce780 2023-06-23T18:04:55.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.787+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218155540 con 0x7fd2180ce780 2023-06-23T18:04:55.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.791+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21804e4c0 con 0x7fd2180ce780 2023-06-23T18:04:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.791+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fd204021030 con 0x7fd2180ce780 2023-06-23T18:04:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.791+0000 7fd2157fa700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 0x7fd1fc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:55.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.791+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6047+0+0 (secure 0 0 0) 0x7fd204085310 con 0x7fd2180ce780 2023-06-23T18:04:55.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.791+0000 7fd21ce2b700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 0x7fd1fc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:55.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.795+0000 7fd21ce2b700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 0x7fd1fc0687a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0045f0 tx=0x7fd20c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:55.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.799+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fd204052810 con 0x7fd2180ce780 2023-06-23T18:04:55.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:55.959+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2180b52b0 con 0x7fd2180ce780 2023-06-23T18:04:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.655+0000 7fd2157fa700 1 -- 172.21.15.22:0/1476625476 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd204055e30 con 0x7fd2180ce780 2023-06-23T18:04:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 msgr2=0x7fd1fc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 0x7fd1fc0687a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0045f0 tx=0x7fd20c008040 comp rx=0 tx=0).stop 2023-06-23T18:04:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 msgr2=0x7fd218153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd204009860 tx=0x7fd204004770 comp rx=0 tx=0).stop 2023-06-23T18:04:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 shutdown_connections 2023-06-23T18:04:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fd1fc0662f0 0x7fd1fc0687a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180cf1b0 0x7fd218144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2180ce780 0x7fd218153fb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 --2- 172.21.15.22:0/1476625476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2180cde70 0x7fd2180c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 >> 172.21.15.22:0/1476625476 conn(0x7fd218069960 msgr2=0x7fd21806fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 shutdown_connections 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.663+0000 7fd21de2d700 1 -- 172.21.15.22:0/1476625476 wait complete. 2023-06-23T18:04:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' removed 2023-06-23T18:04:56.693 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T18:04:56.695 INFO:tasks.ceph.mds.a:Stopping old one... 2023-06-23T18:04:56.695 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-23T18:04:56.696 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:56.696 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T18:04:56.697 INFO:tasks.ceph.mds.c:Stopping old one... 2023-06-23T18:04:56.697 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-23T18:04:56.698 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:56.698 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T18:04:56.699 INFO:tasks.ceph.mds.b:Stopping old one... 2023-06-23T18:04:56.699 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-23T18:04:56.699 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:56.700 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T18:04:56.700 INFO:tasks.ceph.mds.d:Stopping old one... 2023-06-23T18:04:56.700 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-23T18:04:56.700 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:04:56.711 INFO:tasks.ceph.mds.c:Stopped 2023-06-23T18:04:56.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T18:04:56.712 INFO:tasks.ceph.mds.a:Stopped 2023-06-23T18:04:56.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T18:04:56.714 INFO:tasks.ceph.mds.b:Stopped 2023-06-23T18:04:56.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T18:04:56.715 INFO:tasks.ceph.mds.d:Stopped 2023-06-23T18:04:56.716 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T18:04:56.754 INFO:tasks.ceph.mds.c:Started 2023-06-23T18:04:56.755 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:56.756 INFO:tasks.ceph.mds.a:Started 2023-06-23T18:04:56.756 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:56.760 INFO:tasks.ceph.mds.b:Started 2023-06-23T18:04:56.760 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:56.762 INFO:tasks.ceph.mds.d:Started 2023-06-23T18:04:56.762 DEBUG:teuthology.parallel:result is None 2023-06-23T18:04:56.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-06-23T18:04:56.838 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2023-06-23T18:04:56.839 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2023-06-23T18:04:56.869 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-06-23T18:04:56.870 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-06-23T18:04:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 Processor -- start 2023-06-23T18:04:56.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 -- start start 2023-06-23T18:04:56.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4768036770 0x7f4768036b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4768037fe0 con 0x7f47680e47b0 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680370b0 con 0x7f4768037b40 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f4775513700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4768038240 con 0x7f4768036770 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.871+0000 7f476ffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f47680e47b0 0x7f47680e4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50426/0 (socket says 172.21.15.22:50426) 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f476ffff700 1 -- 172.21.15.22:0/2068612023 learned_addr learned my addr 172.21.15.22:0/2068612023 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:56.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f4774d12700 1 --2- 172.21.15.22:0/2068612023 >> v2:172.21.15.22:3300/0 conn(0x7f4768036770 0x7f4768036b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:56.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f4774d12700 1 -- 172.21.15.22:0/2068612023 >> v1:172.21.15.169:6789/0 conn(0x7f4768037b40 legacy=0x7f47680060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f4774d12700 1 -- 172.21.15.22:0/2068612023 >> v1:172.21.15.22:6789/0 conn(0x7f47680e47b0 legacy=0x7f47680e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f4774d12700 1 -- 172.21.15.22:0/2068612023 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680067d0 con 0x7f4768036770 2023-06-23T18:04:56.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.875+0000 7f4774d12700 1 --2- 172.21.15.22:0/2068612023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768036770 0x7f4768036b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4764009850 tx=0x7f4764009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ddb9c89e2afd2a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:56.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f476effd700 1 -- 172.21.15.22:0/2068612023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011830 con 0x7f4768036770 2023-06-23T18:04:56.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f476effd700 1 -- 172.21.15.22:0/2068612023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764011990 con 0x7f4768036770 2023-06-23T18:04:56.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f476effd700 1 -- 172.21.15.22:0/2068612023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011d20 con 0x7f4768036770 2023-06-23T18:04:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- 172.21.15.22:0/2068612023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768036770 msgr2=0x7f4768036b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 --2- 172.21.15.22:0/2068612023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768036770 0x7f4768036b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4764009850 tx=0x7f4764009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- 172.21.15.22:0/2068612023 shutdown_connections 2023-06-23T18:04:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 --2- 172.21.15.22:0/2068612023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768036770 0x7f4768036b70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- 172.21.15.22:0/2068612023 >> 172.21.15.22:0/2068612023 conn(0x7f4768051770 msgr2=0x7f4768051b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- 172.21.15.22:0/2068612023 shutdown_connections 2023-06-23T18:04:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- 172.21.15.22:0/2068612023 wait complete. 2023-06-23T18:04:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 Processor -- start 2023-06-23T18:04:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.879+0000 7f4775513700 1 -- start start 2023-06-23T18:04:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47680e47b0 0x7f47680492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47680d2540 con 0x7f4768037b40 2023-06-23T18:04:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476804a730 con 0x7f47680e47b0 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476804aa10 con 0x7f4768036770 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34198/0 (socket says 172.21.15.22:34198) 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:43334/0 (socket says 172.21.15.22:43334) 2023-06-23T18:04:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 -- 172.21.15.22:0/3868379672 learned_addr learned my addr 172.21.15.22:0/3868379672 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4774d12700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47680e47b0 0x7f47680492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 -- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 msgr2=0x7f476804a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 -- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47680e47b0 msgr2=0x7f47680492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47680e47b0 0x7f47680492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 -- 172.21.15.22:0/3868379672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640094b0 con 0x7f4768037b40 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476ffff700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:04:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476f7fe700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c500 tx=0x7f475c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:56.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0052d0 con 0x7f4768037b40 2023-06-23T18:04:56.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c005430 con 0x7f4768037b40 2023-06-23T18:04:56.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476804acf0 con 0x7f4768037b40 2023-06-23T18:04:56.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0055a0 con 0x7f4768037b40 2023-06-23T18:04:56.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476804b2b0 con 0x7f4768037b40 2023-06-23T18:04:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.883+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476804b4a0 con 0x7f4768037b40 2023-06-23T18:04:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.887+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f475c0038e0 con 0x7f4768037b40 2023-06-23T18:04:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.887+0000 7f476cff9700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 0x7f47540687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.887+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f475c082f00 con 0x7f4768037b40 2023-06-23T18:04:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.887+0000 7f476ffff700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 0x7f47540687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.887+0000 7f476ffff700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 0x7f47540687a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4770067810 tx=0x7f477006d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:56.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:56.895+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f475c050540 con 0x7f4768037b40 2023-06-23T18:04:57.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.071+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4768005530 con 0x7f4768037b40 2023-06-23T18:04:57.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.895+0000 7f476cff9700 1 -- 172.21.15.22:0/3868379672 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f475c053b60 con 0x7f4768037b40 2023-06-23T18:04:57.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 msgr2=0x7f47540687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:57.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 0x7f47540687a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4770067810 tx=0x7f477006d040 comp rx=0 tx=0).stop 2023-06-23T18:04:57.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 msgr2=0x7f47680e23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f475c00c500 tx=0x7f475c00c810 comp rx=0 tx=0).stop 2023-06-23T18:04:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 shutdown_connections 2023-06-23T18:04:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f47540662f0 0x7f47540687a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47680e47b0 0x7f47680492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4768037b40 0x7f47680e23c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:57.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 --2- 172.21.15.22:0/3868379672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4768036770 0x7f476804a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 >> 172.21.15.22:0/3868379672 conn(0x7f4768051770 msgr2=0x7f47680e0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 shutdown_connections 2023-06-23T18:04:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:57.899+0000 7f4775513700 1 -- 172.21.15.22:0/3868379672 wait complete. 2023-06-23T18:04:57.918 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2023-06-23T18:04:57.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-06-23T18:04:58.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.035+0000 7f0305618700 1 Processor -- start 2023-06-23T18:04:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.035+0000 7f0305618700 1 -- start start 2023-06-23T18:04:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0305618700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f03000c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0305618700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0300158610 con 0x7f0300071050 2023-06-23T18:04:58.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0305618700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000ce0f0 con 0x7f03000afe40 2023-06-23T18:04:58.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0305618700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000cc540 con 0x7f0300158240 2023-06-23T18:04:58.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02ff7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0300071050 0x7f030006a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50430/0 (socket says 172.21.15.22:50430) 2023-06-23T18:04:58.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02ff7fe700 1 -- 172.21.15.22:0/1782881040 learned_addr learned my addr 172.21.15.22:0/1782881040 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f03000c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515898440 0 0) 0x7f0300158610 con 0x7f0300071050 2023-06-23T18:04:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e80035d0 con 0x7f0300071050 2023-06-23T18:04:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310955483 0 0) 0x7f02e80035d0 con 0x7f0300071050 2023-06-23T18:04:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 msgr2=0x7f03000c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:58.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f03000c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 >> v1:172.21.15.169:6789/0 conn(0x7f03000afe40 legacy=0x7f030006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000ce390 con 0x7f0300071050 2023-06-23T18:04:58.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f02f4001670 con 0x7f0300071050 2023-06-23T18:04:58.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 >> v1:172.21.15.22:6789/0 conn(0x7f0300071050 legacy=0x7f030006a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f02e8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02e8004870 0x7f02e8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f02e8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000cc540 con 0x7f02e8003a50 2023-06-23T18:04:58.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e800bae0 con 0x7f02e8004870 2023-06-23T18:04:58.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e800bc70 con 0x7f0300158240 2023-06-23T18:04:58.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f02e8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0304e17700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f02e8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02ff7fe700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02e8004870 0x7f02e8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 -- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 msgr2=0x7f02e8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f02e8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 -- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02e8004870 msgr2=0x7f02e8004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02e8004870 0x7f02e8004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f02fffff700 1 -- 172.21.15.22:0/1782881040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e80035d0 con 0x7f02e8003a50 2023-06-23T18:04:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.039+0000 7f0304e17700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158240 0x7f02e8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f02fffff700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f02e8003e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f02f000ab70 tx=0x7f02e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcc56372d30b95b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000f9d0 con 0x7f02e8003a50 2023-06-23T18:04:58.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f000fb30 con 0x7f02e8003a50 2023-06-23T18:04:58.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f02feffd700 1 -- 172.21.15.22:0/1782881040 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000fea0 con 0x7f02e8003a50 2023-06-23T18:04:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- 172.21.15.22:0/1782881040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 msgr2=0x7f02e8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 --2- 172.21.15.22:0/1782881040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f02e8003e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f02f000ab70 tx=0x7f02e80058a0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0304e17700 1 -- 172.21.15.22:0/1782881040 reap_dead start 2023-06-23T18:04:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- 172.21.15.22:0/1782881040 shutdown_connections 2023-06-23T18:04:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- 172.21.15.22:0/1782881040 >> 172.21.15.22:0/1782881040 conn(0x7f03000b1170 msgr2=0x7f030006bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- 172.21.15.22:0/1782881040 shutdown_connections 2023-06-23T18:04:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- 172.21.15.22:0/1782881040 wait complete. 2023-06-23T18:04:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 Processor -- start 2023-06-23T18:04:58.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.043+0000 7f0305618700 1 -- start start 2023-06-23T18:04:58.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f030014bf80 0x7f0300158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158d40 0x7f0300159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300157480 con 0x7f02e8003a50 2023-06-23T18:04:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03001596d0 con 0x7f030014bf80 2023-06-23T18:04:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030014d4e0 con 0x7f0300158d40 2023-06-23T18:04:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34210/0 (socket says 172.21.15.22:34210) 2023-06-23T18:04:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 -- 172.21.15.22:0/304405891 learned_addr learned my addr 172.21.15.22:0/304405891 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02ff7fe700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f030014bf80 0x7f0300158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0304e17700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158d40 0x7f0300159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 -- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158d40 msgr2=0x7f0300159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158d40 0x7f0300159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 -- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f030014bf80 msgr2=0x7f0300158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f030014bf80 0x7f0300158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 -- 172.21.15.22:0/304405891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030014cc20 con 0x7f02e8003a50 2023-06-23T18:04:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fffff700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f02e80047c0 tx=0x7f02f00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000fea0 con 0x7f02e8003a50 2023-06-23T18:04:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f00036c0 con 0x7f02e8003a50 2023-06-23T18:04:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030014ce90 con 0x7f02e8003a50 2023-06-23T18:04:58.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0003a70 con 0x7f02e8003a50 2023-06-23T18:04:58.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.047+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030015b540 con 0x7f02e8003a50 2023-06-23T18:04:58.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030004e4c0 con 0x7f02e8003a50 2023-06-23T18:04:58.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f02f0003bd0 con 0x7f02e8003a50 2023-06-23T18:04:58.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f02fcff9700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 0x7f02ec06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f02f0022070 con 0x7f02e8003a50 2023-06-23T18:04:58.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f02ff7fe700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 0x7f02ec06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.051+0000 7f02ff7fe700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 0x7f02ec06a9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02f4001c90 tx=0x7f02f4007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.055+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f02f00525a0 con 0x7f02e8003a50 2023-06-23T18:04:58.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.279+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0300153020 con 0x7f02e8003a50 2023-06-23T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.279+0000 7f02fcff9700 1 -- 172.21.15.22:0/304405891 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f02f0055bc0 con 0x7f02e8003a50 2023-06-23T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2023-06-23T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2023-06-23T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVk9jlqNBAA0gmLc95g6/e6xHzq2tl3pg==", 2023-06-23T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2023-06-23T18:04:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVkt5xXOxAAAkvAeEOdEzWHtjoiW/O+kw==", 2023-06-23T18:04:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-06-23T18:04:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-06-23T18:04:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2023-06-23T18:04:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVkB2kYOBAAPcbWIvolXCE1AuCCalJxtw==", 2023-06-23T18:04:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-06-23T18:04:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-06-23T18:04:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVk8K5uAxAAPxXvJ10nVt3jVYdr7KXpVw==", 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVkQfyfNhAAwByfToNOhTNRZu4O1IBJzg==", 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBN3pVk/T1qOhAAdhEBQ3HwvuCG7H5y6rDR9w==", 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2023-06-23T18:04:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBR3pVk2GU/BRAA7iy4JAvlQYUQnjY8OXfCpg==", 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBU3pVkRS4PCBAAoU5OdmhHYLDVe3h3ycNW3g==", 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBZ3pVkk7fOMRAA5uciBvSt10WK5EI4Pe94dA==", 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBd3pVkKy2cCBAAWcwFMBYpCQG0uLB+T3fNaQ==", 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBg3pVkj0WwEBAAhnZ8ep9E7VgSGSlW/VzHqg==", 2023-06-23T18:04:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBj3pVk69tvFBAAUlxiBT5pCmO4xTdFvS2xWg==", 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVksM9cBxAAxr2TtBTZIBuP1bjAu3SXFA==", 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-06-23T18:04:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVk/UgFDRAA9SjI8Pvz0kTFtOxQbylsHg==", 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-06-23T18:04:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-06-23T18:04:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-06-23T18:04:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVkuTJEChAAAlJ06AT3zJJ5X1pkUPANtw==", 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-06-23T18:04:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBK3pVkXpoWEBAAxYgwg25L3LtYKfRsc9sI4A==", 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVkBHqWJBAA/ld6gd35pc/K63GqX4+3jQ==", 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mds", 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkBJp+IhAA48a7BmbaYl+y7SYC2UHz0Q==", 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mds" 2023-06-23T18:04:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mgr", 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkd6p+IhAAJioAHXWnHZiaYFBNbWcYNg==", 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mgr" 2023-06-23T18:04:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-osd", 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkRLl+IhAAAuOaLZuwK+A8fUSWxO44JA==", 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-osd" 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd", 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkfsl+IhAAv7HciBbtHGdP+dE0w4/ccw==", 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd" 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkUdh+IhAA66X3S1AcmKj4TbhYsikRuA==", 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rgw", 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBo3pVkuOZ+IhAAuF9EmpMA2Vwl6dEKM83jXQ==", 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rgw" 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVkzdHILxAAYfEFUbh/pXzqk2Al7jpW4Q==", 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2023-06-23T18:04:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBJ3pVkOlQXLRAAo8mydKBw5tYzhARh+AcIog==", 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-06-23T18:04:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-06-23T18:04:58.326 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:04:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 msgr2=0x7f02ec06a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 0x7f02ec06a9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02f4001c90 tx=0x7f02f4007040 comp rx=0 tx=0).stop 2023-06-23T18:04:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 msgr2=0x7f030015ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f02e80047c0 tx=0x7f02f00047e0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 shutdown_connections 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300158d40 0x7f0300159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f02ec068530 0x7f02ec06a9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f030014bf80 0x7f0300158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 --2- 172.21.15.22:0/304405891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02e8003a50 0x7f030015ae80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 >> 172.21.15.22:0/304405891 conn(0x7f03000b1170 msgr2=0x7f0300060430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 shutdown_connections 2023-06-23T18:04:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.283+0000 7f0305618700 1 -- 172.21.15.22:0/304405891 wait complete. 2023-06-23T18:04:58.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-06-23T18:04:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 Processor -- start 2023-06-23T18:04:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 -- start start 2023-06-23T18:04:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe6a4160420 0x7fe6a40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe6a4070040 0x7fe6a40cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a4163ec0 con 0x7fe6a40c0fe0 2023-06-23T18:04:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a4070410 con 0x7fe6a4070040 2023-06-23T18:04:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40cc6f0 con 0x7fe6a4160420 2023-06-23T18:04:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a259c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe6a40c0fe0 0x7fe6a40c13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50436/0 (socket says 172.21.15.22:50436) 2023-06-23T18:04:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a259c700 1 -- 172.21.15.22:0/3490665308 learned_addr learned my addr 172.21.15.22:0/3490665308 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/3490665308 >> v2:172.21.15.22:3300/0 conn(0x7fe6a4070040 0x7fe6a40cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a1d9b700 1 --2- 172.21.15.22:0/3490665308 >> v2:172.21.15.169:3300/0 conn(0x7fe6a4160420 0x7fe6a40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a159a700 1 -- 172.21.15.22:0/3490665308 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461201636 0 0) 0x7fe6a4163ec0 con 0x7fe6a40c0fe0 2023-06-23T18:04:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a159a700 1 -- 172.21.15.22:0/3490665308 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840035d0 con 0x7fe6a40c0fe0 2023-06-23T18:04:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/3490665308 >> v2:172.21.15.169:3300/0 conn(0x7fe6a4160420 msgr2=0x7fe6a40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/3490665308 >> v2:172.21.15.169:3300/0 conn(0x7fe6a4160420 0x7fe6a40cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/3490665308 >> v1:172.21.15.22:6789/0 conn(0x7fe6a40c0fe0 legacy=0x7fe6a40c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/3490665308 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a41641a0 con 0x7fe6a4070040 2023-06-23T18:04:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/3490665308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4070040 0x7fe6a40cc1b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40c85f0 tx=0x7fe69c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1ca9f89543ec7e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a159a700 1 -- 172.21.15.22:0/3490665308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00e070 con 0x7fe6a4070040 2023-06-23T18:04:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a159a700 1 -- 172.21.15.22:0/3490665308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00bae0 con 0x7fe6a4070040 2023-06-23T18:04:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.439+0000 7fe6a159a700 1 -- 172.21.15.22:0/3490665308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00be90 con 0x7fe6a4070040 2023-06-23T18:04:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- 172.21.15.22:0/3490665308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4070040 msgr2=0x7fe6a40cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 --2- 172.21.15.22:0/3490665308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4070040 0x7fe6a40cc1b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40c85f0 tx=0x7fe69c00b330 comp rx=0 tx=0).stop 2023-06-23T18:04:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- 172.21.15.22:0/3490665308 shutdown_connections 2023-06-23T18:04:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 --2- 172.21.15.22:0/3490665308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4070040 0x7fe6a40cc1b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 --2- 172.21.15.22:0/3490665308 >> v2:172.21.15.169:3300/0 conn(0x7fe6a4160420 0x7fe6a40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- 172.21.15.22:0/3490665308 >> 172.21.15.22:0/3490665308 conn(0x7fe6a406a5f0 msgr2=0x7fe6a40caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- 172.21.15.22:0/3490665308 shutdown_connections 2023-06-23T18:04:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- 172.21.15.22:0/3490665308 wait complete. 2023-06-23T18:04:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 Processor -- start 2023-06-23T18:04:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.443+0000 7fe6a359e700 1 -- start start 2023-06-23T18:04:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a4070040 0x7fe6a40c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6a40c0fe0 0x7fe6a40aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40af4b0 con 0x7fe6a4160420 2023-06-23T18:04:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4151020 con 0x7fe6a4070040 2023-06-23T18:04:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c6770 con 0x7fe6a40c0fe0 2023-06-23T18:04:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34236/0 (socket says 172.21.15.22:34236) 2023-06-23T18:04:58.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/1367259338 learned_addr learned my addr 172.21.15.22:0/1367259338 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:58.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a259c700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a4070040 0x7fe6a40c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a1d9b700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6a40c0fe0 0x7fe6a40aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6a40c0fe0 msgr2=0x7fe6a40aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6a40c0fe0 0x7fe6a40aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a4070040 msgr2=0x7fe6a40c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a4070040 0x7fe6a40c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 -- 172.21.15.22:0/1367259338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009a70 con 0x7fe6a4160420 2023-06-23T18:04:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a2d9d700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009f00 tx=0x7fe69c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c004550 con 0x7fe6a4160420 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c020070 con 0x7fe6a4160420 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40c4ab0 con 0x7fe6a4160420 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c004910 con 0x7fe6a4160420 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40c59a0 con 0x7fe6a4160420 2023-06-23T18:04:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.447+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a404e4c0 con 0x7fe6a4160420 2023-06-23T18:04:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.451+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fe69c027480 con 0x7fe6a4160420 2023-06-23T18:04:58.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.451+0000 7fe69affd700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 0x7fe6880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.451+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe69c022070 con 0x7fe6a4160420 2023-06-23T18:04:58.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.451+0000 7fe6a259c700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 0x7fe6880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.451+0000 7fe6a259c700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 0x7fe6880687a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007fd0 tx=0x7fe68c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.459+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fe69c056f60 con 0x7fe6a4160420 2023-06-23T18:04:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.683+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe6a40c68b0 con 0x7fe6a4160420 2023-06-23T18:04:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.691+0000 7fe69affd700 1 -- 172.21.15.22:0/1367259338 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe69c056af0 con 0x7fe6a4160420 2023-06-23T18:04:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 msgr2=0x7fe6880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 0x7fe6880687a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe68c007fd0 tx=0x7fe68c009040 comp rx=0 tx=0).stop 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 msgr2=0x7fe6a40c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009f00 tx=0x7fe69c004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 shutdown_connections 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6a4160420 0x7fe6a40c4570 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6a40c0fe0 0x7fe6a40aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fe6880662f0 0x7fe6880687a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 --2- 172.21.15.22:0/1367259338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a4070040 0x7fe6a40c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 >> 172.21.15.22:0/1367259338 conn(0x7fe6a406a5f0 msgr2=0x7fe6a406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 shutdown_connections 2023-06-23T18:04:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.695+0000 7fe6a359e700 1 -- 172.21.15.22:0/1367259338 wait complete. 2023-06-23T18:04:58.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-06-23T18:04:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.831+0000 7faf24ee7700 1 Processor -- start 2023-06-23T18:04:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.831+0000 7faf24ee7700 1 -- start start 2023-06-23T18:04:58.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf20070280 0x7faf200afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf200c9570 0x7faf2006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf200c9b50 con 0x7faf200affd0 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf201590a0 con 0x7faf200c9570 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf200b0450 con 0x7faf20070280 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf200c9570 0x7faf2006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faf200c9570 0x7faf2006b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34248/0 (socket says 172.21.15.22:34248) 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf17fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7faf200affd0 0x7faf200c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50444/0 (socket says 172.21.15.22:50444) 2023-06-23T18:04:58.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 -- 172.21.15.22:0/478085069 learned_addr learned my addr 172.21.15.22:0/478085069 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:58.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1f7fe700 1 --2- 172.21.15.22:0/478085069 >> v2:172.21.15.169:3300/0 conn(0x7faf20070280 0x7faf200afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 -- 172.21.15.22:0/478085069 >> v2:172.21.15.169:3300/0 conn(0x7faf20070280 msgr2=0x7faf200afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:58.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 --2- 172.21.15.22:0/478085069 >> v2:172.21.15.169:3300/0 conn(0x7faf20070280 0x7faf200afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 -- 172.21.15.22:0/478085069 >> v1:172.21.15.22:6789/0 conn(0x7faf200affd0 legacy=0x7faf200c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 -- 172.21.15.22:0/478085069 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200c9e30 con 0x7faf200c9570 2023-06-23T18:04:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1ffff700 1 --2- 172.21.15.22:0/478085069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf2006b600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faf200cb940 tx=0x7faf1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd0ad76e717703a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1effd700 1 -- 172.21.15.22:0/478085069 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e070 con 0x7faf200c9570 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1effd700 1 -- 172.21.15.22:0/478085069 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1800b910 con 0x7faf200c9570 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf1effd700 1 -- 172.21.15.22:0/478085069 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800bcc0 con 0x7faf200c9570 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- 172.21.15.22:0/478085069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 msgr2=0x7faf2006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 --2- 172.21.15.22:0/478085069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf2006b600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faf200cb940 tx=0x7faf1800b330 comp rx=0 tx=0).stop 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- 172.21.15.22:0/478085069 shutdown_connections 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 --2- 172.21.15.22:0/478085069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf2006b600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 --2- 172.21.15.22:0/478085069 >> v2:172.21.15.169:3300/0 conn(0x7faf20070280 0x7faf200afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- 172.21.15.22:0/478085069 >> 172.21.15.22:0/478085069 conn(0x7faf2006a5f0 msgr2=0x7faf200caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- 172.21.15.22:0/478085069 shutdown_connections 2023-06-23T18:04:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.835+0000 7faf24ee7700 1 -- 172.21.15.22:0/478085069 wait complete. 2023-06-23T18:04:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 Processor -- start 2023-06-23T18:04:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 -- start start 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf20070280 0x7faf2009bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf200affd0 0x7faf20099480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20158c90 con 0x7faf200c9570 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20099b80 con 0x7faf200affd0 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf24ee7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf20098bf0 con 0x7faf20070280 2023-06-23T18:04:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34250/0 (socket says 172.21.15.22:34250) 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf1ffff700 1 -- 172.21.15.22:0/563785071 learned_addr learned my addr 172.21.15.22:0/563785071 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf1f7fe700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf20070280 0x7faf2009bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.839+0000 7faf17fff700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf200affd0 0x7faf20099480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 -- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf20070280 msgr2=0x7faf2009bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf20070280 0x7faf2009bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 -- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf200affd0 msgr2=0x7faf20099480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf200affd0 0x7faf20099480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 -- 172.21.15.22:0/563785071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009b30 con 0x7faf200c9570 2023-06-23T18:04:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1ffff700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf2006dc10 tx=0x7faf18004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18004680 con 0x7faf200c9570 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf180047e0 con 0x7faf200c9570 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20098ed0 con 0x7faf200c9570 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1801b750 con 0x7faf200c9570 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf200ce7a0 con 0x7faf200c9570 2023-06-23T18:04:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf200c2c70 con 0x7faf200c9570 2023-06-23T18:04:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.843+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7faf18021070 con 0x7faf200c9570 2023-06-23T18:04:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.847+0000 7faf1cff9700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 0x7faf040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.847+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7faf18086e40 con 0x7faf200c9570 2023-06-23T18:04:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.847+0000 7faf1f7fe700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 0x7faf040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.847+0000 7faf1f7fe700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 0x7faf040687a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf10009bf0 tx=0x7faf10009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:58.855+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7faf200c2c70 con 0x7faf200c9570 2023-06-23T18:04:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.075+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7faf200c3050 con 0x7faf200c9570 2023-06-23T18:04:59.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.079+0000 7faf1cff9700 1 -- 172.21.15.22:0/563785071 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7faf200c3050 con 0x7faf200c9570 2023-06-23T18:04:59.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 msgr2=0x7faf040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 0x7faf040687a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf10009bf0 tx=0x7faf10009240 comp rx=0 tx=0).stop 2023-06-23T18:04:59.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 msgr2=0x7faf200ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf2006dc10 tx=0x7faf18004af0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 shutdown_connections 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf200c9570 0x7faf200ce260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7faf040662f0 0x7faf040687a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf200affd0 0x7faf20099480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 --2- 172.21.15.22:0/563785071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf20070280 0x7faf2009bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 >> 172.21.15.22:0/563785071 conn(0x7faf2006a5f0 msgr2=0x7faf2006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 shutdown_connections 2023-06-23T18:04:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.083+0000 7faf24ee7700 1 -- 172.21.15.22:0/563785071 wait complete. 2023-06-23T18:04:59.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-06-23T18:04:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 Processor -- start 2023-06-23T18:04:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 -- start start 2023-06-23T18:04:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0c18f0 con 0x7f80cc0cc380 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc163ec0 con 0x7f80cc160420 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d2e9e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc1641a0 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34254/0 (socket says 172.21.15.22:34254) 2023-06-23T18:04:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 -- 172.21.15.22:0/986398007 learned_addr learned my addr 172.21.15.22:0/986398007 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 -- 172.21.15.22:0/986398007 >> v1:172.21.15.169:6789/0 conn(0x7f80cc160420 legacy=0x7f80cc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d0e9a700 1 -- 172.21.15.22:0/986398007 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420905031 0 0) 0x7f80cc0c18f0 con 0x7f80cc0cc380 2023-06-23T18:04:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 -- 172.21.15.22:0/986398007 >> v1:172.21.15.22:6789/0 conn(0x7f80cc0cc380 legacy=0x7f80cc0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 -- 172.21.15.22:0/986398007 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc164480 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d1e9c700 1 --2- 172.21.15.22:0/986398007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009910 tx=0x7f80bc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0828c54b38f865a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d0e9a700 1 -- 172.21.15.22:0/986398007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00ba60 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d0e9a700 1 -- 172.21.15.22:0/986398007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc00bbc0 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.219+0000 7f80d0e9a700 1 -- 172.21.15.22:0/986398007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc009e60 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- 172.21.15.22:0/986398007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 msgr2=0x7f80cc0c13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/986398007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009910 tx=0x7f80bc009c20 comp rx=0 tx=0).stop 2023-06-23T18:04:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- 172.21.15.22:0/986398007 shutdown_connections 2023-06-23T18:04:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/986398007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0c13b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- 172.21.15.22:0/986398007 >> 172.21.15.22:0/986398007 conn(0x7f80cc06a5f0 msgr2=0x7f80cc0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- 172.21.15.22:0/986398007 shutdown_connections 2023-06-23T18:04:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- 172.21.15.22:0/986398007 wait complete. 2023-06-23T18:04:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 Processor -- start 2023-06-23T18:04:59.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.223+0000 7f80d2e9e700 1 -- start start 2023-06-23T18:04:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80cc0cc380 0x7f80cc0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 0x7f80cc15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c3b80 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c3e60 con 0x7f80cc160420 2023-06-23T18:04:59.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0c4140 con 0x7f80cc0cc380 2023-06-23T18:04:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34266/0 (socket says 172.21.15.22:34266) 2023-06-23T18:04:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 -- 172.21.15.22:0/2820288408 learned_addr learned my addr 172.21.15.22:0/2820288408 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d169b700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80cc0cc380 0x7f80cc0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d269d700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 0x7f80cc15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 -- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80cc0cc380 msgr2=0x7f80cc0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80cc0cc380 0x7f80cc0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 -- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 msgr2=0x7f80cc15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 0x7f80cc15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 -- 172.21.15.22:0/2820288408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009570 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d269d700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 0x7f80cc15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d1e9c700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009c60 tx=0x7f80bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0119f0 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc01f070 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc0c4420 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc011d50 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.227+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc0b3c90 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc0c4940 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f80bc01a430 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80c27fc700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 0x7f80b00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f80bc021070 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80d169b700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 0x7f80b00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.231+0000 7f80d169b700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 0x7f80b00687a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c3070 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.239+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f80cc0c4940 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.459+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f80cc0c4940 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.463+0000 7f80c27fc700 1 -- 172.21.15.22:0/2820288408 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f80cc0c4940 con 0x7f80cc0c0fe0 2023-06-23T18:04:59.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 msgr2=0x7f80b00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 0x7f80b00687a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0c3070 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2023-06-23T18:04:59.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 msgr2=0x7f80cc0ac060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009c60 tx=0x7f80bc004770 comp rx=0 tx=0).stop 2023-06-23T18:04:59.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 shutdown_connections 2023-06-23T18:04:59.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f80b00662f0 0x7f80b00687a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc160420 0x7f80cc15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80cc0cc380 0x7f80cc0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 --2- 172.21.15.22:0/2820288408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80cc0c0fe0 0x7f80cc0ac060 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 >> 172.21.15.22:0/2820288408 conn(0x7f80cc06a5f0 msgr2=0x7f80cc0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:59.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 shutdown_connections 2023-06-23T18:04:59.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.467+0000 7f80d2e9e700 1 -- 172.21.15.22:0/2820288408 wait complete. 2023-06-23T18:04:59.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-06-23T18:04:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 Processor -- start 2023-06-23T18:04:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 -- start start 2023-06-23T18:04:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c159c30 0x7f015c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f015c071450 0x7f015c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0b0420 con 0x7f015c0affd0 2023-06-23T18:04:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c06d560 con 0x7f015c071450 2023-06-23T18:04:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c06d0d0 con 0x7f015c159c30 2023-06-23T18:04:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f015c071450 0x7f015c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015a7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f015c0affd0 0x7f015c070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50464/0 (socket says 172.21.15.22:50464) 2023-06-23T18:04:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015a7fc700 1 -- 172.21.15.22:0/3310133929 learned_addr learned my addr 172.21.15.22:0/3310133929 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:04:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f015affd700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c159c30 0x7f015c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.599+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415415781 0 0) 0x7f014c001490 con 0x7f015c0affd0 2023-06-23T18:04:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01440035d0 con 0x7f015c0affd0 2023-06-23T18:04:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015b7fe700 1 -- 172.21.15.22:0/3310133929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c159c30 msgr2=0x7f015c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:04:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015b7fe700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c159c30 0x7f015c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015b7fe700 1 -- 172.21.15.22:0/3310133929 >> v1:172.21.15.22:6789/0 conn(0x7f015c0affd0 legacy=0x7f015c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677759204 0 0) 0x7f01440035d0 con 0x7f015c0affd0 2023-06-23T18:04:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015b7fe700 1 -- 172.21.15.22:0/3310133929 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0718d0 con 0x7f015c071450 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015b7fe700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0cbbe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0154009850 tx=0x7f0154009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9964e148f6ae193 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c001b80 con 0x7f015c071450 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154004030 con 0x7f015c071450 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f0159ffb700 1 -- 172.21.15.22:0/3310133929 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540043e0 con 0x7f015c071450 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 -- 172.21.15.22:0/3310133929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 msgr2=0x7f015c0cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0cbbe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0154009850 tx=0x7f0154009b60 comp rx=0 tx=0).stop 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 -- 172.21.15.22:0/3310133929 shutdown_connections 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0cbbe0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 --2- 172.21.15.22:0/3310133929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c159c30 0x7f015c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 -- 172.21.15.22:0/3310133929 >> 172.21.15.22:0/3310133929 conn(0x7f015c06a5f0 msgr2=0x7f015c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 -- 172.21.15.22:0/3310133929 shutdown_connections 2023-06-23T18:04:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.603+0000 7f015bfff700 1 -- 172.21.15.22:0/3310133929 wait complete. 2023-06-23T18:04:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 Processor -- start 2023-06-23T18:04:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 -- start start 2023-06-23T18:04:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 0x7f015c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 0x7f015c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0b3330 con 0x7f015c071450 2023-06-23T18:04:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15be40 con 0x7f015c159c30 2023-06-23T18:04:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15c120 con 0x7f015c0affd0 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34284/0 (socket says 172.21.15.22:34284) 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 -- 172.21.15.22:0/3476985834 learned_addr learned my addr 172.21.15.22:0/3476985834 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015a7fc700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 0x7f015c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015b7fe700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 0x7f015c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 -- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 msgr2=0x7f015c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 0x7f015c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 -- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 msgr2=0x7f015c15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 0x7f015c15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 -- 172.21.15.22:0/3476985834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540094b0 con 0x7f015c071450 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015a7fc700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 0x7f015c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015b7fe700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 0x7f015c15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015affd700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f015000e840 tx=0x7f015000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500095a0 con 0x7f015c071450 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015000c690 con 0x7f015c071450 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c15c400 con 0x7f015c071450 2023-06-23T18:04:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.607+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015000ca00 con 0x7f015c071450 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c15c9c0 con 0x7f015c071450 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c04e4c0 con 0x7f015c071450 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f015000cb60 con 0x7f015c071450 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f01437fe700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 0x7f01480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f015a7fc700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 0x7f01480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:04:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.611+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0150084300 con 0x7f015c071450 2023-06-23T18:04:59.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.615+0000 7f015a7fc700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 0x7f01480687a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f015c09c0f0 tx=0x7f014c0015c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:04:59.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.619+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f01500518f0 con 0x7f015c071450 2023-06-23T18:04:59.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.843+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f015c0c7ff0 con 0x7f015c071450 2023-06-23T18:04:59.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f01437fe700 1 -- 172.21.15.22:0/3476985834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0150054f10 con 0x7f015c071450 2023-06-23T18:04:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 msgr2=0x7f01480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 0x7f01480687a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f015c09c0f0 tx=0x7f014c0015c0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 msgr2=0x7f015c0b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f015000e840 tx=0x7f015000ec00 comp rx=0 tx=0).stop 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 shutdown_connections 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f01480662f0 0x7f01480687a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f015c159c30 0x7f015c15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f015c0affd0 0x7f015c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 --2- 172.21.15.22:0/3476985834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f015c071450 0x7f015c0b0f90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:04:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 >> 172.21.15.22:0/3476985834 conn(0x7f015c06a5f0 msgr2=0x7f015c0adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:04:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 shutdown_connections 2023-06-23T18:04:59.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.855+0000 7f015bfff700 1 -- 172.21.15.22:0/3476985834 wait complete. 2023-06-23T18:04:59.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-06-23T18:04:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.983+0000 7f3ebffff700 1 Processor -- start 2023-06-23T18:04:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.983+0000 7f3ebffff700 1 -- start start 2023-06-23T18:05:00.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00cc270 con 0x7f3ec00713f0 2023-06-23T18:05:00.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00cc550 con 0x7f3ec00cd310 2023-06-23T18:05:00.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00cc830 con 0x7f3ec01580e0 2023-06-23T18:05:00.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebf7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3ec00713f0 0x7f3ec0158bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50478/0 (socket says 172.21.15.22:50478) 2023-06-23T18:05:00.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebf7fe700 1 -- 172.21.15.22:0/1027435325 learned_addr learned my addr 172.21.15.22:0/1027435325 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:00.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067704235 0 0) 0x7f3ec00cc270 con 0x7f3ec00713f0 2023-06-23T18:05:00.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea80035d0 con 0x7f3ec00713f0 2023-06-23T18:05:00.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995262277 0 0) 0x7f3ec00cc550 con 0x7f3ec00cd310 2023-06-23T18:05:00.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec00cc270 con 0x7f3ec00cd310 2023-06-23T18:05:00.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061284482 0 0) 0x7f3ea80035d0 con 0x7f3ec00713f0 2023-06-23T18:05:00.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 msgr2=0x7f3ec00cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:00.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 >> v1:172.21.15.169:6789/0 conn(0x7f3ec00cd310 legacy=0x7f3ec01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00ccb10 con 0x7f3ec00713f0 2023-06-23T18:05:00.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f3eb80019f0 con 0x7f3ec00713f0 2023-06-23T18:05:00.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 >> v1:172.21.15.22:6789/0 conn(0x7f3ec00713f0 legacy=0x7f3ec0158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ea8003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea8004120 0x7f3ea8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea8004db0 0x7f3ea8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00cc830 con 0x7f3ea8004db0 2023-06-23T18:05:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea800bae0 con 0x7f3ea8004120 2023-06-23T18:05:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea800bca0 con 0x7f3ec01580e0 2023-06-23T18:05:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ea8003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebf7fe700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea8004db0 0x7f3ea8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebe7fc700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea8004120 0x7f3ea8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 -- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea8004120 msgr2=0x7f3ea8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea8004120 0x7f3ea8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 -- 172.21.15.22:0/1027435325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea8004db0 msgr2=0x7f3ea8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea8004db0 0x7f3ea8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 -- 172.21.15.22:0/1027435325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40094b0 con 0x7f3ec01580e0 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebf7fe700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea8004db0 0x7f3ea8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ea8003be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009cf0 tx=0x7f3ea80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4500e600e7f60b05 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004030 con 0x7f3ec01580e0 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb4004190 con 0x7f3ec01580e0 2023-06-23T18:05:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.987+0000 7f3ebdffb700 1 -- 172.21.15.22:0/1027435325 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004540 con 0x7f3ec01580e0 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 msgr2=0x7f3ea8003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 --2- 172.21.15.22:0/1027435325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ea8003be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009cf0 tx=0x7f3ea80058a0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebf7fe700 1 -- 172.21.15.22:0/1027435325 reap_dead start 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- 172.21.15.22:0/1027435325 shutdown_connections 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- 172.21.15.22:0/1027435325 >> 172.21.15.22:0/1027435325 conn(0x7f3ec006a5f0 msgr2=0x7f3ec00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- 172.21.15.22:0/1027435325 shutdown_connections 2023-06-23T18:05:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- 172.21.15.22:0/1027435325 wait complete. 2023-06-23T18:05:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 Processor -- start 2023-06-23T18:05:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- start start 2023-06-23T18:05:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 0x7f3ec00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 0x7f3ec0156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec01445a0 con 0x7f3ec01438c0 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec01442c0 con 0x7f3ec0143dc0 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.991+0000 7f3ebffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00c04c0 con 0x7f3ec01580e0 2023-06-23T18:05:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 0x7f3ec0156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebe7fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 0x7f3ec00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:43416/0 (socket says 172.21.15.22:43416) 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 -- 172.21.15.22:0/435314627 learned_addr learned my addr 172.21.15.22:0/435314627 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 -- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 msgr2=0x7f3ec00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 0x7f3ec00ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 -- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 msgr2=0x7f3ec0156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 0x7f3ec0156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 -- 172.21.15.22:0/435314627 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00c07a0 con 0x7f3ec01580e0 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebe7fc700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 0x7f3ec00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebf7fe700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 0x7f3ec0156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebeffd700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800be30 tx=0x7f3eb40047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4020070 con 0x7f3ec01580e0 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400ee10 con 0x7f3ec01580e0 2023-06-23T18:05:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec00c0a80 con 0x7f3ec01580e0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004030 con 0x7f3ec01580e0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec0157490 con 0x7f3ec01580e0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.995+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec00c0cc0 con 0x7f3ec01580e0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.999+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3eb4003ad0 con 0x7f3ec01580e0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.999+0000 7f3ea77fe700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 0x7f3eac0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.999+0000 7f3ebe7fc700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 0x7f3eac0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.999+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3eb4085050 con 0x7f3ec01580e0 2023-06-23T18:05:00.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:04:59.999+0000 7f3ebe7fc700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 0x7f3eac0687a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0000c00 tx=0x7f3eb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.007+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3ec00c0cc0 con 0x7f3ec01580e0 2023-06-23T18:05:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.231+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3ec00c0c70 con 0x7f3ec01580e0 2023-06-23T18:05:00.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.243+0000 7f3ea77fe700 1 -- 172.21.15.22:0/435314627 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3eb4056320 con 0x7f3ec01580e0 2023-06-23T18:05:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 msgr2=0x7f3eac0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 0x7f3eac0687a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0000c00 tx=0x7f3eb0009040 comp rx=0 tx=0).stop 2023-06-23T18:05:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 msgr2=0x7f3ec00c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800be30 tx=0x7f3eb40047e0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 shutdown_connections 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ec01438c0 0x7f3ec0156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3eac0662f0 0x7f3eac0687a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0143dc0 0x7f3ec00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 --2- 172.21.15.22:0/435314627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec01580e0 0x7f3ec00c0fa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 >> 172.21.15.22:0/435314627 conn(0x7f3ec006a5f0 msgr2=0x7f3ec00ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 shutdown_connections 2023-06-23T18:05:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.247+0000 7f3ebffff700 1 -- 172.21.15.22:0/435314627 wait complete. 2023-06-23T18:05:00.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-06-23T18:05:00.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 Processor -- start 2023-06-23T18:05:00.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 -- start start 2023-06-23T18:05:00.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cf480 0x7fa6d40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6d40cc9e0 0x7fa6d40d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d40b7ec0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d40b1ad0 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d9444700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40d0a10 con 0x7fa6d40cf480 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d3fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa6d40b7af0 0x7fa6d40ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50488/0 (socket says 172.21.15.22:50488) 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d3fff700 1 -- 172.21.15.22:0/1954447726 learned_addr learned my addr 172.21.15.22:0/1954447726 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6d8c43700 1 --2- 172.21.15.22:0/1954447726 >> v2:172.21.15.169:3300/0 conn(0x7fa6d40cc9e0 0x7fa6d40d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.379+0000 7fa6cb7fe700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cf480 0x7fa6d40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d37fe700 1 -- 172.21.15.22:0/1954447726 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021729911 0 0) 0x7fa6d40b7ec0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d37fe700 1 -- 172.21.15.22:0/1954447726 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c00035d0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d8c43700 1 -- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cf480 msgr2=0x7fa6d40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:00.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d8c43700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cf480 0x7fa6d40ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d8c43700 1 -- 172.21.15.22:0/1954447726 >> v1:172.21.15.22:6789/0 conn(0x7fa6d40b7af0 legacy=0x7fa6d40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d8c43700 1 -- 172.21.15.22:0/1954447726 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40d0cb0 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d8c43700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cc9e0 0x7fa6d40d18e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009d90 tx=0x7fa6cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7af31501ee2ea51 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d37fe700 1 -- 172.21.15.22:0/1954447726 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00e070 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d37fe700 1 -- 172.21.15.22:0/1954447726 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00bb20 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d37fe700 1 -- 172.21.15.22:0/1954447726 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc012430 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 -- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cc9e0 msgr2=0x7fa6d40d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cc9e0 0x7fa6d40d18e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009d90 tx=0x7fa6cc00b330 comp rx=0 tx=0).stop 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 -- 172.21.15.22:0/1954447726 shutdown_connections 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cc9e0 0x7fa6d40d18e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 --2- 172.21.15.22:0/1954447726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cf480 0x7fa6d40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 -- 172.21.15.22:0/1954447726 >> 172.21.15.22:0/1954447726 conn(0x7fa6d4069960 msgr2=0x7fa6d4069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 -- 172.21.15.22:0/1954447726 shutdown_connections 2023-06-23T18:05:00.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.383+0000 7fa6d9444700 1 -- 172.21.15.22:0/1954447726 wait complete. 2023-06-23T18:05:00.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 Processor -- start 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- start start 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 0x7fa6d4141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 0x7fa6d415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40c8d90 con 0x7fa6d40b7af0 2023-06-23T18:05:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4142160 con 0x7fa6d40cf480 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d415e650 con 0x7fa6d40cc9e0 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34308/0 (socket says 172.21.15.22:34308) 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 -- 172.21.15.22:0/2834343164 learned_addr learned my addr 172.21.15.22:0/2834343164 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6cb7fe700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 0x7fa6d4141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d8c43700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 0x7fa6d415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 -- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 msgr2=0x7fa6d4141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 0x7fa6d4141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 -- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 msgr2=0x7fa6d415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 0x7fa6d415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 -- 172.21.15.22:0/2834343164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc009a70 con 0x7fa6d40b7af0 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d8c43700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 0x7fa6d415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6cb7fe700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 0x7fa6d4141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d3fff700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c700 tx=0x7fa6c400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4005d90 con 0x7fa6d40b7af0 2023-06-23T18:05:00.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40070d0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d415f170 con 0x7fa6d40b7af0 2023-06-23T18:05:00.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007480 con 0x7fa6d40b7af0 2023-06-23T18:05:00.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.387+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d415ebb0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d404e4c0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa6c40075e0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6d17fa700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 0x7fa6b40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6cb7fe700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 0x7fa6b40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa6c40831d0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.391+0000 7fa6cb7fe700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 0x7fa6b40687a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40ced70 tx=0x7fa6bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.395+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa6c40507d0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.619+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa6d40b4740 con 0x7fa6d40b7af0 2023-06-23T18:05:00.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.627+0000 7fa6d17fa700 1 -- 172.21.15.22:0/2834343164 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa6c4053df0 con 0x7fa6d40b7af0 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 msgr2=0x7fa6b40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 0x7fa6b40687a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40ced70 tx=0x7fa6bc008040 comp rx=0 tx=0).stop 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 msgr2=0x7fa6d40c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c700 tx=0x7fa6c400cac0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 shutdown_connections 2023-06-23T18:05:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa6b40662f0 0x7fa6b40687a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6d40cf480 0x7fa6d415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6d40cc9e0 0x7fa6d4141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 --2- 172.21.15.22:0/2834343164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d40b7af0 0x7fa6d40c8850 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 >> 172.21.15.22:0/2834343164 conn(0x7fa6d4069960 msgr2=0x7fa6d415b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 shutdown_connections 2023-06-23T18:05:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.631+0000 7fa6d9444700 1 -- 172.21.15.22:0/2834343164 wait complete. 2023-06-23T18:05:00.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:05:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 Processor -- start 2023-06-23T18:05:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 -- start start 2023-06-23T18:05:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015f9b0 0x7f30d015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30d00b5210 0x7f30d00b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d00b5b80 con 0x7f30d00cd0d0 2023-06-23T18:05:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d00d04a0 con 0x7f30d00b5210 2023-06-23T18:05:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.763+0000 7f30d62cc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00d0a60 con 0x7f30d015f9b0 2023-06-23T18:05:00.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30d00b5210 0x7f30d00b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30d00b5210 0x7f30d00b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34310/0 (socket says 172.21.15.22:34310) 2023-06-23T18:05:00.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 -- 172.21.15.22:0/2499582074 learned_addr learned my addr 172.21.15.22:0/2499582074 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:00.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d4ac9700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015f9b0 0x7f30d015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 -- 172.21.15.22:0/2499582074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015f9b0 msgr2=0x7f30d015fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015f9b0 0x7f30d015fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 -- 172.21.15.22:0/2499582074 >> v1:172.21.15.169:6789/0 conn(0x7f30d00cd0d0 legacy=0x7f30d00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 -- 172.21.15.22:0/2499582074 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d00cd4a0 con 0x7f30d00b5210 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d5acb700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b5640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009910 tx=0x7f30c8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bab7f8fc1daf441 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30c7fff700 1 -- 172.21.15.22:0/2499582074 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011830 con 0x7f30d00b5210 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30c7fff700 1 -- 172.21.15.22:0/2499582074 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8011990 con 0x7f30d00b5210 2023-06-23T18:05:00.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30c7fff700 1 -- 172.21.15.22:0/2499582074 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011d40 con 0x7f30d00b5210 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 -- 172.21.15.22:0/2499582074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 msgr2=0x7f30d00b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b5640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009910 tx=0x7f30c8009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 -- 172.21.15.22:0/2499582074 shutdown_connections 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b5640 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 --2- 172.21.15.22:0/2499582074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015f9b0 0x7f30d015fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 -- 172.21.15.22:0/2499582074 >> 172.21.15.22:0/2499582074 conn(0x7f30d0069960 msgr2=0x7f30d0069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 -- 172.21.15.22:0/2499582074 shutdown_connections 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.767+0000 7f30d62cc700 1 -- 172.21.15.22:0/2499582074 wait complete. 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 Processor -- start 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- start start 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 0x7f30d01f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 0x7f30d00d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01f5bc0 con 0x7f30d00b5210 2023-06-23T18:05:00.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01f4f40 con 0x7f30d015f9b0 2023-06-23T18:05:00.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01f5220 con 0x7f30d00cd0d0 2023-06-23T18:05:00.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d4ac9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 0x7f30d01f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d4ac9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 0x7f30d01f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:43460/0 (socket says 172.21.15.22:43460) 2023-06-23T18:05:00.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d5acb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 0x7f30d00d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34324/0 (socket says 172.21.15.22:34324) 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d4ac9700 1 -- 172.21.15.22:0/189970994 learned_addr learned my addr 172.21.15.22:0/189970994 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 -- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 msgr2=0x7f30d01f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 0x7f30d01f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 -- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 msgr2=0x7f30d00d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 0x7f30d00d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 -- 172.21.15.22:0/189970994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009570 con 0x7f30d00b5210 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d5acb700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 0x7f30d00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d52ca700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c480 tx=0x7f30c000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00052d0 con 0x7f30d00b5210 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0005430 con 0x7f30d00b5210 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d01f5500 con 0x7f30d00b5210 2023-06-23T18:05:00.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00055a0 con 0x7f30d00b5210 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.771+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d00b4d80 con 0x7f30d00b5210 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.775+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d01f5d00 con 0x7f30d00b5210 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.775+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f30c00038e0 con 0x7f30d00b5210 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.775+0000 7f30c5ffb700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 0x7f30bc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.775+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f30c0082e40 con 0x7f30d00b5210 2023-06-23T18:05:00.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.775+0000 7f30d4ac9700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 0x7f30bc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:00.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.779+0000 7f30d4ac9700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 0x7f30bc0687a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30d00d27a0 tx=0x7f30cc00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:00.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.783+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f30d01f5d00 con 0x7f30d00b5210 2023-06-23T18:05:00.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.947+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30d01f5d00 con 0x7f30d00b5210 2023-06-23T18:05:00.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30c5ffb700 1 -- 172.21.15.22:0/189970994 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+14832 (secure 0 0 0) 0x7f30d01f5d00 con 0x7f30d00b5210 2023-06-23T18:05:00.961 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:00.961 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:04:57.884274+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:05:00.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 msgr2=0x7f30bc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 0x7f30bc0687a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30d00d27a0 tx=0x7f30cc00b040 comp rx=0 tx=0).stop 2023-06-23T18:05:00.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 msgr2=0x7f30d00b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c480 tx=0x7f30c000c790 comp rx=0 tx=0).stop 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 shutdown_connections 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f30bc0662f0 0x7f30bc0687a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d015f9b0 0x7f30d00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d00cd0d0 0x7f30d01f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 --2- 172.21.15.22:0/189970994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d00b5210 0x7f30d00b4840 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:00.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 >> 172.21.15.22:0/189970994 conn(0x7f30d0069960 msgr2=0x7f30d00d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:00.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 shutdown_connections 2023-06-23T18:05:00.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:00.951+0000 7f30d62cc700 1 -- 172.21.15.22:0/189970994 wait complete. 2023-06-23T18:05:00.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:03:31.664428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T18:03:31.695033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:05:00.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:05:01.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.091+0000 7efffdf69700 1 Processor -- start 2023-06-23T18:05:01.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.091+0000 7efffdf69700 1 -- start start 2023-06-23T18:05:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efff80cc430 0x7efff80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 0x7efff806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff815fa00 con 0x7efff806c150 2023-06-23T18:05:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff80cc800 con 0x7efff80cc430 2023-06-23T18:05:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff80b2c60 con 0x7efff80cd870 2023-06-23T18:05:01.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efff80cc430 0x7efff80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efff80cc430 0x7efff80b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34330/0 (socket says 172.21.15.22:34330) 2023-06-23T18:05:01.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 -- 172.21.15.22:0/110100371 learned_addr learned my addr 172.21.15.22:0/110100371 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effeffff700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 0x7efff806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 -- 172.21.15.22:0/110100371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 msgr2=0x7efff806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:01.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 0x7efff806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 -- 172.21.15.22:0/110100371 >> v1:172.21.15.169:6789/0 conn(0x7efff806c150 legacy=0x7efff815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effef7fe700 1 -- 172.21.15.22:0/110100371 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390012359 0 0) 0x7efff815fa00 con 0x7efff806c150 2023-06-23T18:05:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 -- 172.21.15.22:0/110100371 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff815c310 con 0x7efff80cc430 2023-06-23T18:05:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effeffff700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 0x7efff806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffcf67700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff80cc430 0x7efff80b2720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efff4009910 tx=0x7efff4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc7ac85c90195f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effef7fe700 1 -- 172.21.15.22:0/110100371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011830 con 0x7efff80cc430 2023-06-23T18:05:01.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effef7fe700 1 -- 172.21.15.22:0/110100371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4011990 con 0x7efff80cc430 2023-06-23T18:05:01.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7effef7fe700 1 -- 172.21.15.22:0/110100371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011d40 con 0x7efff80cc430 2023-06-23T18:05:01.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- 172.21.15.22:0/110100371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff80cc430 msgr2=0x7efff80b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff80cc430 0x7efff80b2720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efff4009910 tx=0x7efff4009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- 172.21.15.22:0/110100371 shutdown_connections 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cd870 0x7efff806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 --2- 172.21.15.22:0/110100371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff80cc430 0x7efff80b2720 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- 172.21.15.22:0/110100371 >> 172.21.15.22:0/110100371 conn(0x7efff80ccd60 msgr2=0x7efff80cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- 172.21.15.22:0/110100371 shutdown_connections 2023-06-23T18:05:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.095+0000 7efffdf69700 1 -- 172.21.15.22:0/110100371 wait complete. 2023-06-23T18:05:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 Processor -- start 2023-06-23T18:05:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 -- start start 2023-06-23T18:05:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cc430 0x7efff815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efff80cd870 0x7efff81e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff815f8e0 con 0x7efff806c150 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff815ec60 con 0x7efff80cd870 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffdf69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff815ef40 con 0x7efff80cc430 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffcf67700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffcf67700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34346/0 (socket says 172.21.15.22:34346) 2023-06-23T18:05:01.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffcf67700 1 -- 172.21.15.22:0/2508373729 learned_addr learned my addr 172.21.15.22:0/2508373729 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7efffd768700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efff80cd870 0x7efff81e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.099+0000 7effeffff700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cc430 0x7efff815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 -- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cc430 msgr2=0x7efff815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cc430 0x7efff815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 -- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efff80cd870 msgr2=0x7efff81e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efff80cd870 0x7efff81e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 -- 172.21.15.22:0/2508373729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80094b0 con 0x7efff806c150 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffcf67700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efff8072e10 tx=0x7efff4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400b970 con 0x7efff806c150 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff401f070 con 0x7efff806c150 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff4009570 con 0x7efff806c150 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400bcf0 con 0x7efff806c150 2023-06-23T18:05:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff815f500 con 0x7efff806c150 2023-06-23T18:05:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff804e4c0 con 0x7efff806c150 2023-06-23T18:05:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7efff4026480 con 0x7efff806c150 2023-06-23T18:05:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.103+0000 7effed7fa700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 0x7effe40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.107+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7efff4021070 con 0x7efff806c150 2023-06-23T18:05:01.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.107+0000 7effeffff700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 0x7effe40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.107+0000 7effeffff700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 0x7effe40687a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efff80b20f0 tx=0x7effe8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.111+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7efff4052a00 con 0x7efff806c150 2023-06-23T18:05:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efff80b6260 con 0x7efff806c150 2023-06-23T18:05:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7effed7fa700 1 -- 172.21.15.22:0/2508373729 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7efff4056020 con 0x7efff806c150 2023-06-23T18:05:01.283 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-06-23T18:05:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 msgr2=0x7effe40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 0x7effe40687a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efff80b20f0 tx=0x7effe8009240 comp rx=0 tx=0).stop 2023-06-23T18:05:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 msgr2=0x7efff815c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.271+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efff8072e10 tx=0x7efff4004770 comp rx=0 tx=0).stop 2023-06-23T18:05:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 shutdown_connections 2023-06-23T18:05:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7effe40662f0 0x7effe40687a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efff80cd870 0x7efff81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff80cc430 0x7efff815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 --2- 172.21.15.22:0/2508373729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff806c150 0x7efff815c500 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 >> 172.21.15.22:0/2508373729 conn(0x7efff80ccd60 msgr2=0x7efff80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 shutdown_connections 2023-06-23T18:05:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.275+0000 7efffdf69700 1 -- 172.21.15.22:0/2508373729 wait complete. 2023-06-23T18:05:01.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T18:05:01.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.411+0000 7fcc630c5700 1 Processor -- start 2023-06-23T18:05:01.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 -- start start 2023-06-23T18:05:01.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcc5c0af800 0x7fcc5c15e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c0b02a0 0x7fcc5c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c06fe20 con 0x7fcc5c159b40 2023-06-23T18:05:01.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c0b0c00 con 0x7fcc5c0af800 2023-06-23T18:05:01.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc630c5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0afc50 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcc5c0af800 0x7fcc5c15e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcc5c0af800 0x7fcc5c15e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34354/0 (socket says 172.21.15.22:34354) 2023-06-23T18:05:01.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 -- 172.21.15.22:0/3567637036 learned_addr learned my addr 172.21.15.22:0/3567637036 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc628c4700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c0b02a0 0x7fcc5c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 -- 172.21.15.22:0/3567637036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c0b02a0 msgr2=0x7fcc5c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:01.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c0b02a0 0x7fcc5c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 -- 172.21.15.22:0/3567637036 >> v1:172.21.15.169:6789/0 conn(0x7fcc5c159b40 legacy=0x7fcc5c069960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 -- 172.21.15.22:0/3567637036 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0aff00 con 0x7fcc5c0af800 2023-06-23T18:05:01.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc610c1700 1 -- 172.21.15.22:0/3567637036 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551781413 0 0) 0x7fcc5c06fe20 con 0x7fcc5c159b40 2023-06-23T18:05:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc618c2700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0af800 0x7fcc5c15e1b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009910 tx=0x7fcc4c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad51cc10f54e637 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc610c1700 1 -- 172.21.15.22:0/3567637036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c011830 con 0x7fcc5c0af800 2023-06-23T18:05:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.415+0000 7fcc610c1700 1 -- 172.21.15.22:0/3567637036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c011990 con 0x7fcc5c0af800 2023-06-23T18:05:01.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc610c1700 1 -- 172.21.15.22:0/3567637036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c011d40 con 0x7fcc5c0af800 2023-06-23T18:05:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 -- 172.21.15.22:0/3567637036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0af800 msgr2=0x7fcc5c15e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0af800 0x7fcc5c15e1b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009910 tx=0x7fcc4c009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 -- 172.21.15.22:0/3567637036 shutdown_connections 2023-06-23T18:05:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c0b02a0 0x7fcc5c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 --2- 172.21.15.22:0/3567637036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0af800 0x7fcc5c15e1b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 -- 172.21.15.22:0/3567637036 >> 172.21.15.22:0/3567637036 conn(0x7fcc5c06ca60 msgr2=0x7fcc5c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 -- 172.21.15.22:0/3567637036 shutdown_connections 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.419+0000 7fcc630c5700 1 -- 172.21.15.22:0/3567637036 wait complete. 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 Processor -- start 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 -- start start 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c159b40 0x7fcc5c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c14a5e0 0x7fcc5c14aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c153bb0 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c153e50 con 0x7fcc5c14a5e0 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c154130 con 0x7fcc5c159b40 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34358/0 (socket says 172.21.15.22:34358) 2023-06-23T18:05:01.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 -- 172.21.15.22:0/1562581099 learned_addr learned my addr 172.21.15.22:0/1562581099 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc618c2700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c159b40 0x7fcc5c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc628c4700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c14a5e0 0x7fcc5c14aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 -- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c159b40 msgr2=0x7fcc5c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c159b40 0x7fcc5c0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 -- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c14a5e0 msgr2=0x7fcc5c14aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c14a5e0 0x7fcc5c14aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 -- 172.21.15.22:0/1562581099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c009570 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc620c3700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c520 tx=0x7fcc5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580054e0 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc580036a0 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c154410 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.423+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58003a50 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c1549d0 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c154c10 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcc58003bb0 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc52ffd700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 0x7fcc400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcc58083120 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.427+0000 7fcc618c2700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 0x7fcc400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.431+0000 7fcc618c2700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 0x7fcc400687a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c014040 tx=0x7fcc4c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.435+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fcc58050890 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.595+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcc5c154c10 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc52ffd700 1 -- 172.21.15.22:0/1562581099 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcc5c154c10 con 0x7fcc5c0b02a0 2023-06-23T18:05:01.609 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-06-23T18:05:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 msgr2=0x7fcc400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 0x7fcc400687a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c014040 tx=0x7fcc4c008030 comp rx=0 tx=0).stop 2023-06-23T18:05:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 msgr2=0x7fcc5c154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c520 tx=0x7fcc5800c830 comp rx=0 tx=0).stop 2023-06-23T18:05:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 shutdown_connections 2023-06-23T18:05:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcc400662f0 0x7fcc400687a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc5c14a5e0 0x7fcc5c14aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc5c159b40 0x7fcc5c0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 --2- 172.21.15.22:0/1562581099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc5c0b02a0 0x7fcc5c154ef0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 >> 172.21.15.22:0/1562581099 conn(0x7fcc5c0af800 msgr2=0x7fcc5c1526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 shutdown_connections 2023-06-23T18:05:01.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.599+0000 7fcc630c5700 1 -- 172.21.15.22:0/1562581099 wait complete. 2023-06-23T18:05:01.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T18:05:01.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 Processor -- start 2023-06-23T18:05:01.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 -- start start 2023-06-23T18:05:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca600ccd70 0x7fca600c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca60163a50 0x7fca60163e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca600cc360 con 0x7fca6015ff90 2023-06-23T18:05:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca600cdb60 con 0x7fca600ccd70 2023-06-23T18:05:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca674e7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca60160360 con 0x7fca60163a50 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca65ce4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca600ccd70 0x7fca600c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca65ce4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca600ccd70 0x7fca600c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34362/0 (socket says 172.21.15.22:34362) 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca65ce4700 1 -- 172.21.15.22:0/385340375 learned_addr learned my addr 172.21.15.22:0/385340375 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.727+0000 7fca66ce6700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca60163a50 0x7fca60163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca65ce4700 1 -- 172.21.15.22:0/385340375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca60163a50 msgr2=0x7fca60163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca65ce4700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca60163a50 0x7fca60163e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca65ce4700 1 -- 172.21.15.22:0/385340375 >> v1:172.21.15.22:6789/0 conn(0x7fca6015ff90 legacy=0x7fca600c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca65ce4700 1 -- 172.21.15.22:0/385340375 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60164390 con 0x7fca600ccd70 2023-06-23T18:05:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca65ce4700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca600c9f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca48009910 tx=0x7fca48009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6276b5653524cb70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca654e3700 1 -- 172.21.15.22:0/385340375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4800ba60 con 0x7fca600ccd70 2023-06-23T18:05:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca654e3700 1 -- 172.21.15.22:0/385340375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4800bbc0 con 0x7fca600ccd70 2023-06-23T18:05:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca654e3700 1 -- 172.21.15.22:0/385340375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48009e60 con 0x7fca600ccd70 2023-06-23T18:05:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- 172.21.15.22:0/385340375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 msgr2=0x7fca600c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca600c9f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca48009910 tx=0x7fca48009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- 172.21.15.22:0/385340375 shutdown_connections 2023-06-23T18:05:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca60163a50 0x7fca60163e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 --2- 172.21.15.22:0/385340375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca600c9f00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- 172.21.15.22:0/385340375 >> 172.21.15.22:0/385340375 conn(0x7fca600cb2a0 msgr2=0x7fca600cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- 172.21.15.22:0/385340375 shutdown_connections 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- 172.21.15.22:0/385340375 wait complete. 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 Processor -- start 2023-06-23T18:05:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.731+0000 7fca674e7700 1 -- start start 2023-06-23T18:05:01.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca6015ff90 0x7fca600cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 0x7fca600cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60157500 con 0x7fca600ccd70 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca601577e0 con 0x7fca60163a50 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca60157ac0 con 0x7fca6015ff90 2023-06-23T18:05:01.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34374/0 (socket says 172.21.15.22:34374) 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 -- 172.21.15.22:0/3640885190 learned_addr learned my addr 172.21.15.22:0/3640885190 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca65ce4700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca6015ff90 0x7fca600cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca66ce6700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 0x7fca600cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 -- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca6015ff90 msgr2=0x7fca600cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca6015ff90 0x7fca600cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 -- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 msgr2=0x7fca600cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 0x7fca600cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 -- 172.21.15.22:0/3640885190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48009570 con 0x7fca600ccd70 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca66ce6700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 0x7fca600cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca664e5700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c500 tx=0x7fca5000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500052d0 con 0x7fca600ccd70 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca50005430 con 0x7fca600ccd70 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca60157da0 con 0x7fca600ccd70 2023-06-23T18:05:01.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500055a0 con 0x7fca600ccd70 2023-06-23T18:05:01.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.735+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca600c6710 con 0x7fca600ccd70 2023-06-23T18:05:01.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca60157f90 con 0x7fca600ccd70 2023-06-23T18:05:01.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fca500038e0 con 0x7fca600ccd70 2023-06-23T18:05:01.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca5effd700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 0x7fca400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:01.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca65ce4700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 0x7fca400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:01.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fca50082f40 con 0x7fca600ccd70 2023-06-23T18:05:01.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.739+0000 7fca65ce4700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 0x7fca400687a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca48000c00 tx=0x7fca48008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:01.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.747+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fca500505d0 con 0x7fca600ccd70 2023-06-23T18:05:01.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.911+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fca60157640 con 0x7fca600ccd70 2023-06-23T18:05:01.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.911+0000 7fca5effd700 1 -- 172.21.15.22:0/3640885190 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+126 (secure 0 0 0) 0x7fca50017090 con 0x7fca600ccd70 2023-06-23T18:05:01.924 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:01.924 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2023-06-23T18:05:01.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:05:01.924 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2023-06-23T18:05:01.924 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 2, 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": "rbd" 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:05:01.925 INFO:teuthology.orchestra.run.smithi022.stdout:] 2023-06-23T18:05:01.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 msgr2=0x7fca400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 0x7fca400687a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca48000c00 tx=0x7fca48008030 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 msgr2=0x7fca601582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c500 tx=0x7fca5000c810 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 shutdown_connections 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fca400662f0 0x7fca400687a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60163a50 0x7fca600cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca6015ff90 0x7fca600cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 --2- 172.21.15.22:0/3640885190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca600ccd70 0x7fca601582c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:01.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 >> 172.21.15.22:0/3640885190 conn(0x7fca600cb2a0 msgr2=0x7fca60159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:01.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 shutdown_connections 2023-06-23T18:05:01.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:01.915+0000 7fca674e7700 1 -- 172.21.15.22:0/3640885190 wait complete. 2023-06-23T18:05:01.942 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T18:05:01.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T18:05:02.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.055+0000 7f2cc359e700 1 Processor -- start 2023-06-23T18:05:02.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.055+0000 7f2cc359e700 1 -- start start 2023-06-23T18:05:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cbc036c20 0x7f2cbc053930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc053e70 con 0x7f2cbc0367d0 2023-06-23T18:05:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc054110 con 0x7f2cbc036c20 2023-06-23T18:05:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0543f0 con 0x7f2cbc0df750 2023-06-23T18:05:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cbc036c20 0x7f2cbc053930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cbc036c20 0x7f2cbc053930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34382/0 (socket says 172.21.15.22:34382) 2023-06-23T18:05:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 -- 172.21.15.22:0/1175202996 learned_addr learned my addr 172.21.15.22:0/1175202996 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc259c700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 -- 172.21.15.22:0/1175202996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 msgr2=0x7f2cbc007710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc007710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 -- 172.21.15.22:0/1175202996 >> v1:172.21.15.169:6789/0 conn(0x7f2cbc0367d0 legacy=0x7f2cbc038ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 -- 172.21.15.22:0/1175202996 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0546d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc2d9d700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc053930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4078870 tx=0x7f2cc40674a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613268e964433a4b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc159a700 1 -- 172.21.15.22:0/1175202996 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4073810 con 0x7f2cbc036c20 2023-06-23T18:05:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc159a700 1 -- 172.21.15.22:0/1175202996 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4073970 con 0x7f2cbc036c20 2023-06-23T18:05:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc159a700 1 -- 172.21.15.22:0/1175202996 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc406da30 con 0x7f2cbc036c20 2023-06-23T18:05:02.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- 172.21.15.22:0/1175202996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 msgr2=0x7f2cbc053930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc053930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4078870 tx=0x7f2cc40674a0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- 172.21.15.22:0/1175202996 shutdown_connections 2023-06-23T18:05:02.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc053930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1175202996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc007710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- 172.21.15.22:0/1175202996 >> 172.21.15.22:0/1175202996 conn(0x7f2cbc0517d0 msgr2=0x7f2cbc051ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- 172.21.15.22:0/1175202996 shutdown_connections 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.059+0000 7f2cc359e700 1 -- 172.21.15.22:0/1175202996 wait complete. 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 Processor -- start 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 -- start start 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0367d0 0x7f2cbc17ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc1688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc16c8b0 con 0x7f2cbc036c20 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0dfb20 con 0x7f2cbc0367d0 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc16afc0 con 0x7f2cbc0df750 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34388/0 (socket says 172.21.15.22:34388) 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 -- 172.21.15.22:0/1892137378 learned_addr learned my addr 172.21.15.22:0/1892137378 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc259c700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0367d0 0x7f2cbc17ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc2d9d700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc1688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 -- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 msgr2=0x7f2cbc1688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc1688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 -- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0367d0 msgr2=0x7f2cbc17ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0367d0 0x7f2cbc17ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 -- 172.21.15.22:0/1892137378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc16b2a0 con 0x7f2cbc036c20 2023-06-23T18:05:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.063+0000 7f2cc1d9b700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c530 tx=0x7f2cb400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40052d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4005430 con 0x7f2cbc036c20 2023-06-23T18:05:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc16b580 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40055a0 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc16b860 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04d7d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2cb40038e0 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 0x7f2ca40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2cb4082ea0 con 0x7f2cbc036c20 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.067+0000 7f2cc259c700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 0x7f2ca40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.071+0000 7f2cc259c700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 0x7f2ca40687a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc035cd0 tx=0x7f2cb8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.079+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2cbc04d7d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.243+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2cbc04d7d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.695+0000 7f2cb2ffd700 1 -- 172.21.15.22:0/1892137378 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2cbc04d7d0 con 0x7f2cbc036c20 2023-06-23T18:05:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 msgr2=0x7f2ca40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 0x7f2ca40687a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc035cd0 tx=0x7f2cb8009240 comp rx=0 tx=0).stop 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 msgr2=0x7f2cbc16a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c530 tx=0x7f2cb400c840 comp rx=0 tx=0).stop 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 shutdown_connections 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0df750 0x7f2cbc1688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cbc036c20 0x7f2cbc16a650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2ca40662f0 0x7f2ca40687a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 --2- 172.21.15.22:0/1892137378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0367d0 0x7f2cbc17ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.703+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 >> 172.21.15.22:0/1892137378 conn(0x7f2cbc0517d0 msgr2=0x7f2cbc052900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.707+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 shutdown_connections 2023-06-23T18:05:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.707+0000 7f2cc359e700 1 -- 172.21.15.22:0/1892137378 wait complete. 2023-06-23T18:05:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2023-06-23T18:05:02.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T18:05:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 Processor -- start 2023-06-23T18:05:02.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 -- start start 2023-06-23T18:05:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43700cca20 0x7f43700d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43700d21d0 0x7f43700d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700d1d40 con 0x7f43700b1920 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700b1cf0 con 0x7f43700d21d0 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f43754c7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700cf130 con 0x7f43700cca20 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f4374cc6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43700d21d0 0x7f43700d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f436ffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f43700b1920 0x7f43700cffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50506/0 (socket says 172.21.15.22:50506) 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.847+0000 7f4374cc6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f43700d21d0 0x7f43700d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34398/0 (socket says 172.21.15.22:34398) 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436ffff700 1 -- 172.21.15.22:0/968380725 learned_addr learned my addr 172.21.15.22:0/968380725 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436f7fe700 1 --2- 172.21.15.22:0/968380725 >> v2:172.21.15.169:3300/0 conn(0x7f43700cca20 0x7f43700d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f4374cc6700 1 -- 172.21.15.22:0/968380725 >> v2:172.21.15.169:3300/0 conn(0x7f43700cca20 msgr2=0x7f43700d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f4374cc6700 1 --2- 172.21.15.22:0/968380725 >> v2:172.21.15.169:3300/0 conn(0x7f43700cca20 0x7f43700d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f4374cc6700 1 -- 172.21.15.22:0/968380725 >> v1:172.21.15.22:6789/0 conn(0x7f43700b1920 legacy=0x7f43700cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436effd700 1 -- 172.21.15.22:0/968380725 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775656572 0 0) 0x7f43700d1d40 con 0x7f43700b1920 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f4374cc6700 1 -- 172.21.15.22:0/968380725 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700cf410 con 0x7f43700d21d0 2023-06-23T18:05:02.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f4374cc6700 1 --2- 172.21.15.22:0/968380725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700d21d0 0x7f43700d2630 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f43700b4020 tx=0x7f436800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2481792a4e6e2ce2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436effd700 1 -- 172.21.15.22:0/968380725 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436800e070 con 0x7f43700d21d0 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436effd700 1 -- 172.21.15.22:0/968380725 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800bbd0 con 0x7f43700d21d0 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f436effd700 1 -- 172.21.15.22:0/968380725 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368012430 con 0x7f43700d21d0 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 -- 172.21.15.22:0/968380725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700d21d0 msgr2=0x7f43700d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 --2- 172.21.15.22:0/968380725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700d21d0 0x7f43700d2630 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f43700b4020 tx=0x7f436800b330 comp rx=0 tx=0).stop 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 -- 172.21.15.22:0/968380725 shutdown_connections 2023-06-23T18:05:02.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 --2- 172.21.15.22:0/968380725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700d21d0 0x7f43700d2630 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 --2- 172.21.15.22:0/968380725 >> v2:172.21.15.169:3300/0 conn(0x7f43700cca20 0x7f43700d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 -- 172.21.15.22:0/968380725 >> 172.21.15.22:0/968380725 conn(0x7f4370069960 msgr2=0x7f4370069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:02.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 -- 172.21.15.22:0/968380725 shutdown_connections 2023-06-23T18:05:02.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.851+0000 7f43754c7700 1 -- 172.21.15.22:0/968380725 wait complete. 2023-06-23T18:05:02.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 Processor -- start 2023-06-23T18:05:02.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 -- start start 2023-06-23T18:05:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43700cca20 0x7f43701522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43700d21d0 0x7f43700c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b5200 con 0x7f43700b1920 2023-06-23T18:05:02.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700cb370 con 0x7f43700cca20 2023-06-23T18:05:02.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f43754c7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b5450 con 0x7f43700d21d0 2023-06-23T18:05:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f436ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f436ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34402/0 (socket says 172.21.15.22:34402) 2023-06-23T18:05:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.855+0000 7f436ffff700 1 -- 172.21.15.22:0/2598185126 learned_addr learned my addr 172.21.15.22:0/2598185126 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f4374cc6700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43700d21d0 0x7f43700c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436f7fe700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43700cca20 0x7f43701522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 -- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43700d21d0 msgr2=0x7f43700c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43700d21d0 0x7f43700c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 -- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43700cca20 msgr2=0x7f43701522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43700cca20 0x7f43701522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 -- 172.21.15.22:0/2598185126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368009a70 con 0x7f43700b1920 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436ffff700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f436000c580 tx=0x7f436000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360004ca0 con 0x7f43700b1920 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360004e20 con 0x7f43700b1920 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43700b5730 con 0x7f43700b1920 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360007310 con 0x7f43700b1920 2023-06-23T18:05:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4370150d90 con 0x7f43700b1920 2023-06-23T18:05:02.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.859+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437004e4c0 con 0x7f43700b1920 2023-06-23T18:05:02.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.863+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f4360007470 con 0x7f43700b1920 2023-06-23T18:05:02.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.863+0000 7f436cff9700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 0x7f435c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.863+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f4360083a70 con 0x7f43700b1920 2023-06-23T18:05:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.863+0000 7f436f7fe700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 0x7f435c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.863+0000 7f436f7fe700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 0x7f435c0687a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43700b3c70 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:02.867+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f4360050f80 con 0x7f43700b1920 2023-06-23T18:05:03.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.031+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f43700b46d0 con 0x7f43700b1920 2023-06-23T18:05:03.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.703+0000 7f436cff9700 1 -- 172.21.15.22:0/2598185126 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4360058e60 con 0x7f43700b1920 2023-06-23T18:05:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 msgr2=0x7f435c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 0x7f435c0687a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43700b3c70 tx=0x7f4364008040 comp rx=0 tx=0).stop 2023-06-23T18:05:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 msgr2=0x7f4370150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f436000c580 tx=0x7f436000c940 comp rx=0 tx=0).stop 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 shutdown_connections 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43700d21d0 0x7f43700c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f435c0662f0 0x7f435c0687a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43700cca20 0x7f43701522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 --2- 172.21.15.22:0/2598185126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43700b1920 0x7f4370150850 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 >> 172.21.15.22:0/2598185126 conn(0x7f4370069960 msgr2=0x7f43700ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 shutdown_connections 2023-06-23T18:05:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.727+0000 7f43754c7700 1 -- 172.21.15.22:0/2598185126 wait complete. 2023-06-23T18:05:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2023-06-23T18:05:03.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T18:05:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.859+0000 7f217ac69700 1 Processor -- start 2023-06-23T18:05:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.859+0000 7f217ac69700 1 -- start start 2023-06-23T18:05:03.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f21740d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740ccb30 con 0x7f21740cf3a0 2023-06-23T18:05:03.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740d2880 con 0x7f21740d2360 2023-06-23T18:05:03.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740d2b60 con 0x7f21740d1f30 2023-06-23T18:05:03.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2179c67700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f21740cf3a0 0x7f21740cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50516/0 (socket says 172.21.15.22:50516) 2023-06-23T18:05:03.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2179c67700 1 -- 172.21.15.22:0/4263897345 learned_addr learned my addr 172.21.15.22:0/4263897345 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:03.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2179466700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f21740d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468283517 0 0) 0x7f21740ccb30 con 0x7f21740cf3a0 2023-06-23T18:05:03.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21600035d0 con 0x7f21740cf3a0 2023-06-23T18:05:03.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107846303 0 0) 0x7f21740d2880 con 0x7f21740d2360 2023-06-23T18:05:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21740ccb30 con 0x7f21740d2360 2023-06-23T18:05:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1068673670 0 0) 0x7f21600035d0 con 0x7f21740cf3a0 2023-06-23T18:05:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 msgr2=0x7f21740d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f21740d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 >> v1:172.21.15.169:6789/0 conn(0x7f21740d2360 legacy=0x7f21740ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740d2e40 con 0x7f21740cf3a0 2023-06-23T18:05:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3505207057 0 0) 0x7f21700017f0 con 0x7f21740cf3a0 2023-06-23T18:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 >> v1:172.21.15.22:6789/0 conn(0x7f21740cf3a0 legacy=0x7f21740cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f2160003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2160004140 0x7f2160004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160004db0 0x7f2160005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740d2b60 con 0x7f2160004db0 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216000bae0 con 0x7f2160004140 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216000bca0 con 0x7f21740d1f30 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160004db0 0x7f2160005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2179c67700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f2160003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2179466700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2160004140 0x7f2160004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 -- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 msgr2=0x7f2160003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21740d1f30 0x7f2160003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 -- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2160004140 msgr2=0x7f2160004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2160004140 0x7f2160004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 -- 172.21.15.22:0/4263897345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600035d0 con 0x7f2160004db0 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217a468700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160004db0 0x7f2160005220 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f216c002230 tx=0x7f21600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f11385bead2122c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c001f30 con 0x7f2160004db0 2023-06-23T18:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00ce80 con 0x7f2160004db0 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f2178c65700 1 -- 172.21.15.22:0/4263897345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c003c60 con 0x7f2160004db0 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 -- 172.21.15.22:0/4263897345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160004db0 msgr2=0x7f2160005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.863+0000 7f217ac69700 1 --2- 172.21.15.22:0/4263897345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160004db0 0x7f2160005220 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f216c002230 tx=0x7f21600058a0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 -- 172.21.15.22:0/4263897345 reap_dead start 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/4263897345 shutdown_connections 2023-06-23T18:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/4263897345 >> 172.21.15.22:0/4263897345 conn(0x7f217406a280 msgr2=0x7f217409b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/4263897345 shutdown_connections 2023-06-23T18:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/4263897345 wait complete. 2023-06-23T18:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 Processor -- start 2023-06-23T18:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- start start 2023-06-23T18:05:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 0x7f21740b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174158cb0 0x7f21740b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740ce110 con 0x7f21740cd750 2023-06-23T18:05:03.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217415e2e0 con 0x7f2174158cb0 2023-06-23T18:05:03.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740b6800 con 0x7f21741588e0 2023-06-23T18:05:03.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f2179c67700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 0x7f21740b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f2179c67700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 0x7f21740b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:43584/0 (socket says 172.21.15.22:43584) 2023-06-23T18:05:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f2179c67700 1 -- 172.21.15.22:0/328243874 learned_addr learned my addr 172.21.15.22:0/328243874 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34430/0 (socket says 172.21.15.22:34430) 2023-06-23T18:05:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f2179466700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174158cb0 0x7f21740b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 -- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 msgr2=0x7f21740b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 0x7f21740b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 -- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174158cb0 msgr2=0x7f21740b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:03.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174158cb0 0x7f21740b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:03.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 -- 172.21.15.22:0/328243874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740b6ae0 con 0x7f21740cd750 2023-06-23T18:05:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217a468700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f216c014000 tx=0x7f216c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c003e30 con 0x7f21740cd750 2023-06-23T18:05:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0050d0 con 0x7f21740cd750 2023-06-23T18:05:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740b6dc0 con 0x7f21740cd750 2023-06-23T18:05:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c005480 con 0x7f21740cd750 2023-06-23T18:05:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.867+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21740c9a20 con 0x7f21740cd750 2023-06-23T18:05:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217404e4c0 con 0x7f21740cd750 2023-06-23T18:05:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f216c019040 con 0x7f21740cd750 2023-06-23T18:05:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f21667fc700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 0x7f21580687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f216c0854c0 con 0x7f21740cd750 2023-06-23T18:05:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f2179c67700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 0x7f21580687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.871+0000 7f2179c67700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 0x7f21580687a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21740ce670 tx=0x7f2170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:03.879+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f216c052af0 con 0x7f21740cd750 2023-06-23T18:05:04.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.087+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f217406a680 con 0x7f21740cd750 2023-06-23T18:05:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.095+0000 7f21667fc700 1 -- 172.21.15.22:0/328243874 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f216c0170c0 con 0x7f21740cd750 2023-06-23T18:05:04.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.099+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 msgr2=0x7f21580687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.099+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 0x7f21580687a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21740ce670 tx=0x7f2170009040 comp rx=0 tx=0).stop 2023-06-23T18:05:04.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 msgr2=0x7f21740cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f216c014000 tx=0x7f216c001650 comp rx=0 tx=0).stop 2023-06-23T18:05:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 shutdown_connections 2023-06-23T18:05:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21740cd750 0x7f21740cdbd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f21580662f0 0x7f21580687a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174158cb0 0x7f21740b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 --2- 172.21.15.22:0/328243874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21741588e0 0x7f21740b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 >> 172.21.15.22:0/328243874 conn(0x7f217406a280 msgr2=0x7f21740b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 shutdown_connections 2023-06-23T18:05:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.103+0000 7f217ac69700 1 -- 172.21.15.22:0/328243874 wait complete. 2023-06-23T18:05:04.143 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 6 and data pool 7 2023-06-23T18:05:04.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:05:04.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 Processor -- start 2023-06-23T18:05:04.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 -- start start 2023-06-23T18:05:04.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 0x7f61e00cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f61e0070040 0x7f61e00cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e0163ec0 con 0x7f61e00c0fe0 2023-06-23T18:05:04.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e00cc6f0 con 0x7f61e0070040 2023-06-23T18:05:04.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61dffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e01641a0 con 0x7f61e0160420 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61deffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61e00c0fe0 0x7f61e00c13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50520/0 (socket says 172.21.15.22:50520) 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61deffd700 1 -- 172.21.15.22:0/301589672 learned_addr learned my addr 172.21.15.22:0/301589672 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61de7fc700 1 --2- 172.21.15.22:0/301589672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 0x7f61e00cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61df7fe700 1 --2- 172.21.15.22:0/301589672 >> v2:172.21.15.169:3300/0 conn(0x7f61e0070040 0x7f61e00cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61ddffb700 1 -- 172.21.15.22:0/301589672 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413707594 0 0) 0x7f61e0163ec0 con 0x7f61e00c0fe0 2023-06-23T18:05:04.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.251+0000 7f61ddffb700 1 -- 172.21.15.22:0/301589672 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c80035d0 con 0x7f61e00c0fe0 2023-06-23T18:05:04.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61de7fc700 1 -- 172.21.15.22:0/301589672 >> v2:172.21.15.169:3300/0 conn(0x7f61e0070040 msgr2=0x7f61e00cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61de7fc700 1 --2- 172.21.15.22:0/301589672 >> v2:172.21.15.169:3300/0 conn(0x7f61e0070040 0x7f61e00cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61de7fc700 1 -- 172.21.15.22:0/301589672 >> v1:172.21.15.22:6789/0 conn(0x7f61e00c0fe0 legacy=0x7f61e00c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61de7fc700 1 -- 172.21.15.22:0/301589672 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0164480 con 0x7f61e0160420 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61de7fc700 1 --2- 172.21.15.22:0/301589672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 0x7f61e00cbc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009d90 tx=0x7f61d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b1de5dc03a7d3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61ddffb700 1 -- 172.21.15.22:0/301589672 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000e070 con 0x7f61e0160420 2023-06-23T18:05:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61ddffb700 1 -- 172.21.15.22:0/301589672 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000bb20 con 0x7f61e0160420 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61ddffb700 1 -- 172.21.15.22:0/301589672 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0012430 con 0x7f61e0160420 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- 172.21.15.22:0/301589672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 msgr2=0x7f61e00cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 --2- 172.21.15.22:0/301589672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 0x7f61e00cbc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009d90 tx=0x7f61d000b330 comp rx=0 tx=0).stop 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- 172.21.15.22:0/301589672 shutdown_connections 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 --2- 172.21.15.22:0/301589672 >> v2:172.21.15.169:3300/0 conn(0x7f61e0070040 0x7f61e00cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 --2- 172.21.15.22:0/301589672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0160420 0x7f61e00cbc70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- 172.21.15.22:0/301589672 >> 172.21.15.22:0/301589672 conn(0x7f61e006a5f0 msgr2=0x7f61e00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- 172.21.15.22:0/301589672 shutdown_connections 2023-06-23T18:05:04.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- 172.21.15.22:0/301589672 wait complete. 2023-06-23T18:05:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 Processor -- start 2023-06-23T18:05:04.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.255+0000 7f61dffff700 1 -- start start 2023-06-23T18:05:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0070040 0x7f61e00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 0x7f61e01f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e01f5a00 con 0x7f61e00c0fe0 2023-06-23T18:05:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00b1870 con 0x7f61e0160420 2023-06-23T18:05:04.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0099f90 con 0x7f61e0070040 2023-06-23T18:05:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34438/0 (socket says 172.21.15.22:34438) 2023-06-23T18:05:04.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 -- 172.21.15.22:0/986522213 learned_addr learned my addr 172.21.15.22:0/986522213 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61df7fe700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 0x7f61e01f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61deffd700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0070040 0x7f61e00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 -- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0070040 msgr2=0x7f61e00b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0070040 0x7f61e00b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 -- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 msgr2=0x7f61e01f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 0x7f61e01f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 -- 172.21.15.22:0/986522213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0009a70 con 0x7f61e00c0fe0 2023-06-23T18:05:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61df7fe700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 0x7f61e01f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61de7fc700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61e01642e0 tx=0x7f61d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0012430 con 0x7f61e00c0fe0 2023-06-23T18:05:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0020070 con 0x7f61e00c0fe0 2023-06-23T18:05:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e009a270 con 0x7f61e00c0fe0 2023-06-23T18:05:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0004280 con 0x7f61e00c0fe0 2023-06-23T18:05:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e009a830 con 0x7f61e00c0fe0 2023-06-23T18:05:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.259+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e0163ec0 con 0x7f61e00c0fe0 2023-06-23T18:05:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.263+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f61d00043e0 con 0x7f61e00c0fe0 2023-06-23T18:05:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.263+0000 7f61c7fff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 0x7f61cc0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.263+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f61d0022070 con 0x7f61e00c0fe0 2023-06-23T18:05:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.263+0000 7f61deffd700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 0x7f61cc0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.263+0000 7f61deffd700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 0x7f61cc0687a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61d40011e0 tx=0x7f61d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.267+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f61d0052e70 con 0x7f61e00c0fe0 2023-06-23T18:05:04.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.435+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e00cf160 con 0x7f61e00c0fe0 2023-06-23T18:05:04.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.435+0000 7f61c7fff700 1 -- 172.21.15.22:0/986522213 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18501 (secure 0 0 0) 0x7f61d0056490 con 0x7f61e00c0fe0 2023-06-23T18:05:04.448 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:04.448 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:05:04.091082+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:05:02.285334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-23T18:05:03.077987+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:05:04.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 msgr2=0x7f61cc0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 0x7f61cc0687a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61d40011e0 tx=0x7f61d4009040 comp rx=0 tx=0).stop 2023-06-23T18:05:04.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 msgr2=0x7f61e00b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61e01642e0 tx=0x7f61d0004af0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 shutdown_connections 2023-06-23T18:05:04.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f61cc0662f0 0x7f61cc0687a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0160420 0x7f61e01f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e00c0fe0 0x7f61e00b2480 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 --2- 172.21.15.22:0/986522213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0070040 0x7f61e00b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 >> 172.21.15.22:0/986522213 conn(0x7f61e006a5f0 msgr2=0x7f61e015f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 shutdown_connections 2023-06-23T18:05:04.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.439+0000 7f61dffff700 1 -- 172.21.15.22:0/986522213 wait complete. 2023-06-23T18:05:04.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:05:04.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.567+0000 7f7dbb00e700 1 Processor -- start 2023-06-23T18:05:04.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- start start 2023-06-23T18:05:04.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7dac03a450 0x7f7dac04f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac03a820 con 0x7f7dac0e2d80 2023-06-23T18:05:04.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0e3210 con 0x7f7dac036770 2023-06-23T18:05:04.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0060c0 con 0x7f7dac03a450 2023-06-23T18:05:04.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db980b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7dac0e2d80 0x7f7dac0528c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50534/0 (socket says 172.21.15.22:50534) 2023-06-23T18:05:04.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db980b700 1 -- 172.21.15.22:0/898239169 learned_addr learned my addr 172.21.15.22:0/898239169 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:04.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dba00c700 1 --2- 172.21.15.22:0/898239169 >> v2:172.21.15.169:3300/0 conn(0x7f7dac03a450 0x7f7dac04f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db900a700 1 -- 172.21.15.22:0/898239169 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847982857 0 0) 0x7f7dac03a820 con 0x7f7dac0e2d80 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db900a700 1 -- 172.21.15.22:0/898239169 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da00035d0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dba00c700 1 -- 172.21.15.22:0/898239169 >> v1:172.21.15.169:6789/0 conn(0x7f7dac036770 legacy=0x7f7dac0064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dba00c700 1 -- 172.21.15.22:0/898239169 >> v1:172.21.15.22:6789/0 conn(0x7f7dac0e2d80 legacy=0x7f7dac0528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dba00c700 1 -- 172.21.15.22:0/898239169 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0e34f0 con 0x7f7dac03a450 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dba00c700 1 --2- 172.21.15.22:0/898239169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac03a450 0x7f7dac04f340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009850 tx=0x7f7db0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ccaa8b3471748 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db900a700 1 -- 172.21.15.22:0/898239169 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011830 con 0x7f7dac03a450 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db900a700 1 -- 172.21.15.22:0/898239169 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db0011990 con 0x7f7dac03a450 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7db900a700 1 -- 172.21.15.22:0/898239169 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011d00 con 0x7f7dac03a450 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- 172.21.15.22:0/898239169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac03a450 msgr2=0x7f7dac04f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/898239169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac03a450 0x7f7dac04f340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009850 tx=0x7f7db0009b60 comp rx=0 tx=0).stop 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- 172.21.15.22:0/898239169 shutdown_connections 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/898239169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac03a450 0x7f7dac04f340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- 172.21.15.22:0/898239169 >> 172.21.15.22:0/898239169 conn(0x7f7dac051770 msgr2=0x7f7dac051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- 172.21.15.22:0/898239169 shutdown_connections 2023-06-23T18:05:04.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.571+0000 7f7dbb00e700 1 -- 172.21.15.22:0/898239169 wait complete. 2023-06-23T18:05:04.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 Processor -- start 2023-06-23T18:05:04.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 -- start start 2023-06-23T18:05:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 0x7f7dac0d7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dac03a450 0x7f7dac0e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac04c1a0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac04c480 con 0x7f7dac036770 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dbb00e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac04c760 con 0x7f7dac03a450 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34446/0 (socket says 172.21.15.22:34446) 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba00c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 0x7f7dac0d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 -- 172.21.15.22:0/592405307 learned_addr learned my addr 172.21.15.22:0/592405307 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:04.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7db980b700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dac03a450 0x7f7dac0e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 -- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dac03a450 msgr2=0x7f7dac0e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dac03a450 0x7f7dac0e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 -- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 msgr2=0x7f7dac0d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 0x7f7dac0d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 -- 172.21.15.22:0/592405307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00094b0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba00c700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 0x7f7dac0d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.575+0000 7f7dba80d700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7db40653d0 tx=0x7f7db404e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4066a80 con 0x7f7dac0e2d80 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4066be0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac04ca40 con 0x7f7dac0e2d80 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db406d0e0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac04bdc0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac04cf60 con 0x7f7dac0e2d80 2023-06-23T18:05:04.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f7db406d260 con 0x7f7dac0e2d80 2023-06-23T18:05:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 0x7f7d980687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7db40fcaf0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.579+0000 7f7dba00c700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 0x7f7d980687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.583+0000 7f7dba00c700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 0x7f7d980687a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7db00090b0 tx=0x7f7db0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.587+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f7db40cd8b0 con 0x7f7dac0e2d80 2023-06-23T18:05:04.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.751+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dac005570 con 0x7f7dac0e2d80 2023-06-23T18:05:04.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.751+0000 7f7daaffd700 1 -- 172.21.15.22:0/592405307 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18501 (secure 0 0 0) 0x7f7db40cd440 con 0x7f7dac0e2d80 2023-06-23T18:05:04.764 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:04.764 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:05:04.091082+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:05:02.285334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-23T18:05:03.077987+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:05:04.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 msgr2=0x7f7d980687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 0x7f7d980687a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7db00090b0 tx=0x7f7db0008030 comp rx=0 tx=0).stop 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 msgr2=0x7f7dac034f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7db40653d0 tx=0x7f7db404e250 comp rx=0 tx=0).stop 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 shutdown_connections 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dac0e2d80 0x7f7dac034f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dac03a450 0x7f7dac0e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f7d980662f0 0x7f7d980687a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 --2- 172.21.15.22:0/592405307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dac036770 0x7f7dac0d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 >> 172.21.15.22:0/592405307 conn(0x7f7dac051770 msgr2=0x7f7dac034750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 shutdown_connections 2023-06-23T18:05:04.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.755+0000 7f7dbb00e700 1 -- 172.21.15.22:0/592405307 wait complete. 2023-06-23T18:05:04.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T18:05:04.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 Processor -- start 2023-06-23T18:05:04.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 -- start start 2023-06-23T18:05:04.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f585815d0a0 0x7f58580af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d470 0x7f585806f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585815da40 con 0x7f585806f7a0 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580cb640 con 0x7f585815d0a0 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f585d756700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580cc370 con 0x7f585815d470 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.883+0000 7f5856ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f585815d0a0 0x7f58580af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f58577fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f585806f7a0 0x7f585815ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50540/0 (socket says 172.21.15.22:50540) 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f58577fe700 1 -- 172.21.15.22:0/986889624 learned_addr learned my addr 172.21.15.22:0/986889624 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f58567fc700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d470 0x7f585806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5855ffb700 1 -- 172.21.15.22:0/986889624 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870858417 0 0) 0x7f585815da40 con 0x7f585806f7a0 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5855ffb700 1 -- 172.21.15.22:0/986889624 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58400035d0 con 0x7f585806f7a0 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5856ffd700 1 -- 172.21.15.22:0/986889624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d470 msgr2=0x7f585806f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5856ffd700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d470 0x7f585806f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5856ffd700 1 -- 172.21.15.22:0/986889624 >> v1:172.21.15.22:6789/0 conn(0x7f585806f7a0 legacy=0x7f585815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5856ffd700 1 -- 172.21.15.22:0/986889624 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580cc650 con 0x7f585815d0a0 2023-06-23T18:05:04.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5856ffd700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585815d0a0 0x7f58580af890 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5848009850 tx=0x7f5848009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3abb34f6c803fe server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5855ffb700 1 -- 172.21.15.22:0/986889624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848011830 con 0x7f585815d0a0 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5855ffb700 1 -- 172.21.15.22:0/986889624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848011990 con 0x7f585815d0a0 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f5855ffb700 1 -- 172.21.15.22:0/986889624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848011d40 con 0x7f585815d0a0 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 -- 172.21.15.22:0/986889624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585815d0a0 msgr2=0x7f58580af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585815d0a0 0x7f58580af890 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5848009850 tx=0x7f5848009b60 comp rx=0 tx=0).stop 2023-06-23T18:05:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 -- 172.21.15.22:0/986889624 shutdown_connections 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d470 0x7f585806f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 --2- 172.21.15.22:0/986889624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585815d0a0 0x7f58580af890 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 -- 172.21.15.22:0/986889624 >> 172.21.15.22:0/986889624 conn(0x7f58580cdb30 msgr2=0x7f58580d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 -- 172.21.15.22:0/986889624 shutdown_connections 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.887+0000 7f585d756700 1 -- 172.21.15.22:0/986889624 wait complete. 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 Processor -- start 2023-06-23T18:05:04.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- start start 2023-06-23T18:05:04.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 0x7f5858157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f585815d470 0x7f5858159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858156fe0 con 0x7f585806f7a0 2023-06-23T18:05:04.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580ca140 con 0x7f585815d470 2023-06-23T18:05:04.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c98a0 con 0x7f585815d0a0 2023-06-23T18:05:04.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34456/0 (socket says 172.21.15.22:34456) 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 -- 172.21.15.22:0/4177667944 learned_addr learned my addr 172.21.15.22:0/4177667944 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f58567fc700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 0x7f5858157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f58577fe700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f585815d470 0x7f5858159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 -- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 msgr2=0x7f5858157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 0x7f5858157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 -- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f585815d470 msgr2=0x7f5858159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:04.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f585815d470 0x7f5858159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 -- 172.21.15.22:0/4177667944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480094b0 con 0x7f585806f7a0 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f58567fc700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 0x7f5858157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f5856ffd700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58580b3010 tx=0x7f5848004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848004480 con 0x7f585806f7a0 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584801f070 con 0x7f585806f7a0 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580c9b80 con 0x7f585806f7a0 2023-06-23T18:05:04.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848003900 con 0x7f585806f7a0 2023-06-23T18:05:04.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.891+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58581e3300 con 0x7f585806f7a0 2023-06-23T18:05:04.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585804dc70 con 0x7f585806f7a0 2023-06-23T18:05:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f5848021030 con 0x7f585806f7a0 2023-06-23T18:05:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f583f7fe700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 0x7f58440687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5848085380 con 0x7f585806f7a0 2023-06-23T18:05:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f58567fc700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 0x7f58440687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.895+0000 7f58567fc700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 0x7f58440687a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58581f2560 tx=0x7f584c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:04.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:04.903+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f5848052870 con 0x7f585806f7a0 2023-06-23T18:05:05.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:05.103+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f58580ccbe0 con 0x7f585806f7a0 2023-06-23T18:05:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.111+0000 7f583f7fe700 1 -- 172.21.15.22:0/4177667944 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5848006ce0 con 0x7f585806f7a0 2023-06-23T18:05:06.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 msgr2=0x7f58440687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 0x7f58440687a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58581f2560 tx=0x7f584c008040 comp rx=0 tx=0).stop 2023-06-23T18:05:06.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 msgr2=0x7f58581497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58580b3010 tx=0x7f5848004770 comp rx=0 tx=0).stop 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 shutdown_connections 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f58440662f0 0x7f58440687a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f585815d470 0x7f5858159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585815d0a0 0x7f5858157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 --2- 172.21.15.22:0/4177667944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f585806f7a0 0x7f58581497c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 >> 172.21.15.22:0/4177667944 conn(0x7f58580cdb30 msgr2=0x7f5858072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:06.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 shutdown_connections 2023-06-23T18:05:06.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.115+0000 7f585d756700 1 -- 172.21.15.22:0/4177667944 wait complete. 2023-06-23T18:05:06.141 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-23T18:05:06.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T18:05:06.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 Processor -- start 2023-06-23T18:05:06.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 -- start start 2023-06-23T18:05:06.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 0x7f2a3c0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0ccb30 con 0x7f2a3c0d2360 2023-06-23T18:05:06.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0cfcb0 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a40d5a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0d2880 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a32ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 0x7f2a3c0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3bfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2a3c0d2360 0x7f2a3c15f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:57964/0 (socket says 172.21.15.22:57964) 2023-06-23T18:05:06.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1864042633 >> v2:172.21.15.169:3300/0 conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:48830/0 (socket says 172.21.15.22:48830) 2023-06-23T18:05:06.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3bfff700 1 -- 172.21.15.22:0/1864042633 learned_addr learned my addr 172.21.15.22:0/1864042633 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 -- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 msgr2=0x7f2a3c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 0x7f2a3c0d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 -- 172.21.15.22:0/1864042633 >> v1:172.21.15.169:6789/0 conn(0x7f2a3c0d2360 legacy=0x7f2a3c15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a3b7fe700 1 -- 172.21.15.22:0/1864042633 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0d2b60 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.263+0000 7f2a32ffd700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 0x7f2a3c0d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009840 tx=0x7f2a2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc0c7efa409ab896 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:06.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a3affd700 1 -- 172.21.15.22:0/1864042633 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c005e50 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a3affd700 1 -- 172.21.15.22:0/1864042633 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c006030 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a3affd700 1 -- 172.21.15.22:0/1864042633 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0063c0 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0cf3a0 msgr2=0x7f2a3c0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009840 tx=0x7f2a2c009b50 comp rx=0 tx=0).stop 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1864042633 shutdown_connections 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0d1f30 0x7f2a3c0d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1864042633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0cf3a0 0x7f2a3c0cf770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1864042633 >> 172.21.15.22:0/1864042633 conn(0x7f2a3c06a280 msgr2=0x7f2a3c09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:06.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1864042633 shutdown_connections 2023-06-23T18:05:06.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1864042633 wait complete. 2023-06-23T18:05:06.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 Processor -- start 2023-06-23T18:05:06.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.267+0000 7f2a40d5a700 1 -- start start 2023-06-23T18:05:06.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0cf3a0 0x7f2a3c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 0x7f2a3c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b8190 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b7bd0 con 0x7f2a3c0d2360 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b7eb0 con 0x7f2a3c0cf3a0 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34468/0 (socket says 172.21.15.22:34468) 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 -- 172.21.15.22:0/1679401865 learned_addr learned my addr 172.21.15.22:0/1679401865 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0cf3a0 0x7f2a3c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a3bfff700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 0x7f2a3c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 -- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0cf3a0 msgr2=0x7f2a3c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0cf3a0 0x7f2a3c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 -- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 msgr2=0x7f2a3c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:06.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 0x7f2a3c15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:06.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 -- 172.21.15.22:0/1679401865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240094b0 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a3bfff700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 0x7f2a3c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:06.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a32ffd700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400ebe0 tx=0x7f2a2400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400c890 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400c9f0 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c009450 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400cd80 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0c99a0 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.271+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c0c9500 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.275+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f2a24013470 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.275+0000 7f2a38ff9700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 0x7f2a1c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:06.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.275+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2a24083f70 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.275+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 0x7f2a1c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:06.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.275+0000 7f2a3b7fe700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 0x7f2a1c0687a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009b90 tx=0x7f2a2c008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.283+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f2a240b4700 con 0x7f2a3c0d1f30 2023-06-23T18:05:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:06.483+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a3c04e4c0 con 0x7f2a3c0d1f30 2023-06-23T18:05:07.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.371+0000 7f2a38ff9700 1 -- 172.21.15.22:0/1679401865 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a240512c0 con 0x7f2a3c0d1f30 2023-06-23T18:05:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 msgr2=0x7f2a1c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 0x7f2a1c0687a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009b90 tx=0x7f2a2c008070 comp rx=0 tx=0).stop 2023-06-23T18:05:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 msgr2=0x7f2a3c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400ebe0 tx=0x7f2a2400eef0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 shutdown_connections 2023-06-23T18:05:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f2a1c0662f0 0x7f2a1c0687a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a3c0d2360 0x7f2a3c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a3c0d1f30 0x7f2a3c0b7180 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 --2- 172.21.15.22:0/1679401865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a3c0cf3a0 0x7f2a3c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 >> 172.21.15.22:0/1679401865 conn(0x7f2a3c06a280 msgr2=0x7f2a3c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 shutdown_connections 2023-06-23T18:05:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.375+0000 7f2a40d5a700 1 -- 172.21.15.22:0/1679401865 wait complete. 2023-06-23T18:05:07.405 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-23T18:05:07.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:05:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 Processor -- start 2023-06-23T18:05:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- start start 2023-06-23T18:05:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa9240cf480 0x7fa9240ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9240d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240b7ec0 con 0x7fa9240b7af0 2023-06-23T18:05:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9240b1ad0 con 0x7fa9240cf480 2023-06-23T18:05:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240d0a10 con 0x7fa9240cc9e0 2023-06-23T18:05:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa9240cf480 0x7fa9240ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa9237fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa9240b7af0 0x7fa9240ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34566/0 (socket says 172.21.15.22:34566) 2023-06-23T18:05:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa9237fe700 1 -- 172.21.15.22:0/4229683181 learned_addr learned my addr 172.21.15.22:0/4229683181 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa923fff700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9240d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 -- 172.21.15.22:0/4229683181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 msgr2=0x7fa9240d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9240d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 -- 172.21.15.22:0/4229683181 >> v1:172.21.15.22:6789/0 conn(0x7fa9240b7af0 legacy=0x7fa9240ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 -- 172.21.15.22:0/4229683181 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240d0cb0 con 0x7fa9240cf480 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa922ffd700 1 -- 172.21.15.22:0/4229683181 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248097986 0 0) 0x7fa9240b7ec0 con 0x7fa9240b7af0 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa91bfff700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240ce540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009850 tx=0x7fa90c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60582a5777286d37 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa922ffd700 1 -- 172.21.15.22:0/4229683181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c011830 con 0x7fa9240cf480 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa922ffd700 1 -- 172.21.15.22:0/4229683181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c011990 con 0x7fa9240cf480 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa922ffd700 1 -- 172.21.15.22:0/4229683181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c011d20 con 0x7fa9240cf480 2023-06-23T18:05:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- 172.21.15.22:0/4229683181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 msgr2=0x7fa9240ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240ce540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009850 tx=0x7fa90c009b60 comp rx=0 tx=0).stop 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- 172.21.15.22:0/4229683181 shutdown_connections 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9240d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 --2- 172.21.15.22:0/4229683181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240ce540 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- 172.21.15.22:0/4229683181 >> 172.21.15.22:0/4229683181 conn(0x7fa924069960 msgr2=0x7fa924069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- 172.21.15.22:0/4229683181 shutdown_connections 2023-06-23T18:05:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.523+0000 7fa928a68700 1 -- 172.21.15.22:0/4229683181 wait complete. 2023-06-23T18:05:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.527+0000 7fa928a68700 1 Processor -- start 2023-06-23T18:05:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.527+0000 7fa928a68700 1 -- start start 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.527+0000 7fa928a68700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9240b7af0 0x7fa924158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.527+0000 7fa928a68700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9241523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.527+0000 7fa928a68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240c9820 con 0x7fa9240cf480 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa924159300 con 0x7fa9240b7af0 2023-06-23T18:05:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92415d8d0 con 0x7fa9240cc9e0 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44530/0 (socket says 172.21.15.22:44530) 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 -- 172.21.15.22:0/3736614589 learned_addr learned my addr 172.21.15.22:0/3736614589 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa9237fe700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9240b7af0 0x7fa924158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa91bfff700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9241523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 -- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 msgr2=0x7fa9241523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9241523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 -- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9240b7af0 msgr2=0x7fa924158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9240b7af0 0x7fa924158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 -- 172.21.15.22:0/3736614589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0094b0 con 0x7fa9240cf480 2023-06-23T18:05:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa923fff700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa91c007e40 tx=0x7fa91c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00eea0 con 0x7fa9240cf480 2023-06-23T18:05:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c014420 con 0x7fa9240cf480 2023-06-23T18:05:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92415cfa0 con 0x7fa9240cf480 2023-06-23T18:05:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0147b0 con 0x7fa9240cf480 2023-06-23T18:05:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92415d560 con 0x7fa9240cf480 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa924073810 con 0x7fa9240cf480 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa91c014910 con 0x7fa9240cf480 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 0x7fa9040687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.531+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa91c084720 con 0x7fa9240cf480 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.535+0000 7fa9237fe700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 0x7fa9040687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.535+0000 7fa9237fe700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 0x7fa9040687a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914007c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.539+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fa91c0519c0 con 0x7fa9240cf480 2023-06-23T18:05:07.682 INFO:tasks.ceph.mds.c.smithi022.stderr:2023-06-23T18:05:07.671+0000 7f9664326700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:05:07.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.735+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92404e4c0 con 0x7fa9240cf480 2023-06-23T18:05:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.735+0000 7fa920ff9700 1 -- 172.21.15.22:0/3736614589 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fa91c054fe0 con 0x7fa9240cf480 2023-06-23T18:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1697006502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":1697006502},{"type":"v1","addr":"172.21.15.22:6837","nonce":1697006502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2478537796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2478537796},{"type":"v1","addr":"172.21.15.169:6834","nonce":2478537796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/880910702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":880910702},{"type":"v1","addr":"172.21.15.169:6835","nonce":880910702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:05:04.092062+0000","modified":"2023-06-23T18:05:07.368660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6836/3366896912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3366896912},{"type":"v1","addr":"172.21.15.22:6836","nonce":3366896912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:05:07.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.735+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 msgr2=0x7fa9040687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.735+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 0x7fa9040687a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914007c20 comp rx=0 tx=0).stop 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 msgr2=0x7fa9240cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa91c007e40 tx=0x7fa91c00e3c0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 shutdown_connections 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9240cf480 0x7fa9240cb840 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9240cc9e0 0x7fa9241523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fa9040662f0 0x7fa9040687a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 --2- 172.21.15.22:0/3736614589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9240b7af0 0x7fa924158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 >> 172.21.15.22:0/3736614589 conn(0x7fa924069960 msgr2=0x7fa92415b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:07.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 shutdown_connections 2023-06-23T18:05:07.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.739+0000 7fa928a68700 1 -- 172.21.15.22:0/3736614589 wait complete. 2023-06-23T18:05:07.750 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2023-06-23T18:05:07.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:05:07.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 Processor -- start 2023-06-23T18:05:07.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- start start 2023-06-23T18:05:07.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0b0420 con 0x7fcfac071450 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac06d0d0 con 0x7fcfac0affd0 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac06d560 con 0x7fcfac159c30 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb116f700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcfac071450 0x7fcfac0cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34580/0 (socket says 172.21.15.22:34580) 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb116f700 1 -- 172.21.15.22:0/3633005205 learned_addr learned my addr 172.21.15.22:0/3633005205 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb096e700 1 --2- 172.21.15.22:0/3633005205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158780205 0 0) 0x7fcfa4001490 con 0x7fcfac071450 2023-06-23T18:05:07.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf9c0035d0 con 0x7fcfac071450 2023-06-23T18:05:07.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978066810 0 0) 0x7fcfac06d0d0 con 0x7fcfac0affd0 2023-06-23T18:05:07.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa4001490 con 0x7fcfac0affd0 2023-06-23T18:05:07.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb096e700 1 -- 172.21.15.22:0/3633005205 >> v1:172.21.15.169:6789/0 conn(0x7fcfac0affd0 legacy=0x7fcfac070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb096e700 1 -- 172.21.15.22:0/3633005205 >> v1:172.21.15.22:6789/0 conn(0x7fcfac071450 legacy=0x7fcfac0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb096e700 1 -- 172.21.15.22:0/3633005205 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0718d0 con 0x7fcfac159c30 2023-06-23T18:05:07.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb096e700 1 --2- 172.21.15.22:0/3633005205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0afa90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009880 tx=0x7fcfa0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f32faf6db29b486 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011830 con 0x7fcfac159c30 2023-06-23T18:05:07.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0011990 con 0x7fcfac159c30 2023-06-23T18:05:07.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfab7fe700 1 -- 172.21.15.22:0/3633005205 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011d00 con 0x7fcfac159c30 2023-06-23T18:05:07.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- 172.21.15.22:0/3633005205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 msgr2=0x7fcfac0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 --2- 172.21.15.22:0/3633005205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0afa90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009880 tx=0x7fcfa0009b90 comp rx=0 tx=0).stop 2023-06-23T18:05:07.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- 172.21.15.22:0/3633005205 shutdown_connections 2023-06-23T18:05:07.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 --2- 172.21.15.22:0/3633005205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0afa90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- 172.21.15.22:0/3633005205 >> 172.21.15.22:0/3633005205 conn(0x7fcfac06a5f0 msgr2=0x7fcfac0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- 172.21.15.22:0/3633005205 shutdown_connections 2023-06-23T18:05:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.867+0000 7fcfb1970700 1 -- 172.21.15.22:0/3633005205 wait complete. 2023-06-23T18:05:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 Processor -- start 2023-06-23T18:05:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 -- start start 2023-06-23T18:05:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1430e0 con 0x7fcfac071450 2023-06-23T18:05:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac158b00 con 0x7fcfac0affd0 2023-06-23T18:05:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb1970700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac158260 con 0x7fcfac159c30 2023-06-23T18:05:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb096e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46790/0 (socket says 172.21.15.22:46790) 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 -- 172.21.15.22:0/2991431578 learned_addr learned my addr 172.21.15.22:0/2991431578 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb096e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44536/0 (socket says 172.21.15.22:44536) 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb116f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 -- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 msgr2=0x7fcfac0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 -- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 msgr2=0x7fcfac155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfabfff700 1 -- 172.21.15.22:0/2991431578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00094b0 con 0x7fcfac0affd0 2023-06-23T18:05:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.871+0000 7fcfb096e700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:07.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfabfff700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800c520 tx=0x7fcf9800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980052d0 con 0x7fcfac0affd0 2023-06-23T18:05:07.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf98005450 con 0x7fcfac0affd0 2023-06-23T18:05:07.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac158540 con 0x7fcfac0affd0 2023-06-23T18:05:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980038e0 con 0x7fcfac0affd0 2023-06-23T18:05:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac15b300 con 0x7fcfac0affd0 2023-06-23T18:05:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac04e4c0 con 0x7fcfac0affd0 2023-06-23T18:05:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.875+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fcf98003a40 con 0x7fcfac0affd0 2023-06-23T18:05:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.879+0000 7fcfa97fa700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 0x7fcf900687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.879+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcf980832e0 con 0x7fcfac0affd0 2023-06-23T18:05:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.879+0000 7fcfb096e700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 0x7fcf900687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.879+0000 7fcfb096e700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 0x7fcf900687a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00090b0 tx=0x7fcfa0010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:07.883+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fcf98050790 con 0x7fcfac0affd0 2023-06-23T18:05:08.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.051+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfac06ca70 con 0x7fcfac0affd0 2023-06-23T18:05:08.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.055+0000 7fcfa97fa700 1 -- 172.21.15.22:0/2991431578 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18488 (secure 0 0 0) 0x7fcf98053db0 con 0x7fcfac0affd0 2023-06-23T18:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":39,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","created":"2023-06-23T18:03:20.577480+0000","modified":"2023-06-23T18:05:05.096458+0000","last_up_change":"2023-06-23T18:03:31.218404+0000","last_in_change":"2023-06-23T18:03:22.997726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:03:31.664428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T18:03:31.695033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:05:02.285334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-23T18:05:03.077987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91bf5ca8-1094-4e50-969f-08f055225329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6810","nonce":445321366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6814","nonce":445321366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6822","nonce":445321366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":445321366},{"type":"v1","addr":"172.21.15.22:6818","nonce":445321366}]},"public_addr":"172.21.15.22:6810/445321366","cluster_addr":"172.21.15.22:6814/445321366","heartbeat_back_addr":"172.21.15.22:6822/445321366","heartbeat_front_addr":"172.21.15.22:6818/445321366","state":["exists","up"]},{"osd":1,"uuid":"bb186bf3-9d2b-4443-bf14-011526a0d3ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6811","nonce":3293388284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6815","nonce":3293388284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6821","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6823","nonce":3293388284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":3293388284},{"type":"v1","addr":"172.21.15.22:6819","nonce":3293388284}]},"public_addr":"172.21.15.22:6811/3293388284","cluster_addr":"172.21.15.22:6815/3293388284","heartbeat_back_addr":"172.21.15.22:6823/3293388284","heartbeat_front_addr":"172.21.15.22:6819/3293388284","state":["exists","up"]},{"osd":2,"uuid":"230da105-5aaa-46f0-85dd-38990a357b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6801","nonce":3001036907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6803","nonce":3001036907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6807","nonce":3001036907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3001036907},{"type":"v1","addr":"172.21.15.22:6805","nonce":3001036907}]},"public_addr":"172.21.15.22:6801/3001036907","cluster_addr":"172.21.15.22:6803/3001036907","heartbeat_back_addr":"172.21.15.22:6807/3001036907","heartbeat_front_addr":"172.21.15.22:6805/3001036907","state":["exists","up"]},{"osd":3,"uuid":"81d76dd6-d216-41d6-991e-b4c32bea0c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6825","nonce":1664577612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6827","nonce":1664577612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6831","nonce":1664577612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1664577612},{"type":"v1","addr":"172.21.15.22:6829","nonce":1664577612}]},"public_addr":"172.21.15.22:6825/1664577612","cluster_addr":"172.21.15.22:6827/1664577612","heartbeat_back_addr":"172.21.15.22:6831/1664577612","heartbeat_front_addr":"172.21.15.22:6829/1664577612","state":["exists","up"]},{"osd":4,"uuid":"f344bea8-74a9-462e-b206-9d6a1d53b973","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6808","nonce":1771639099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6811","nonce":1771639099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6815","nonce":1771639099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1771639099},{"type":"v1","addr":"172.21.15.169:6813","nonce":1771639099}]},"public_addr":"172.21.15.169:6808/1771639099","cluster_addr":"172.21.15.169:6811/1771639099","heartbeat_back_addr":"172.21.15.169:6815/1771639099","heartbeat_front_addr":"172.21.15.169:6813/1771639099","state":["exists","up"]},{"osd":5,"uuid":"9ac9c71a-a05a-4486-a6bd-9870e66f848a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6817","nonce":1436778973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6819","nonce":1436778973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6823","nonce":1436778973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1436778973},{"type":"v1","addr":"172.21.15.169:6821","nonce":1436778973}]},"public_addr":"172.21.15.169:6817/1436778973","cluster_addr":"172.21.15.169:6819/1436778973","heartbeat_back_addr":"172.21.15.169:6823/1436778973","heartbeat_front_addr":"172.21.15.169:6821/1436778973","state":["exists","up"]},{"osd":6,"uuid":"0afa3f52-08ce-48a6-bd59-ca0fe0f4ea05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6825","nonce":3045974108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6827","nonce":3045974108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6831","nonce":3045974108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3045974108},{"type":"v1","addr":"172.21.15.169:6829","nonce":3045974108}]},"public_addr":"172.21.15.169:6825/3045974108","cluster_addr":"172.21.15.169:6827/3045974108","heartbeat_back_addr":"172.21.15.169:6831/3045974108","heartbeat_front_addr":"172.21.15.169:6829/3045974108","state":["exists","up"]},{"osd":7,"uuid":"1a6c4715-ddad-4f26-a63d-62554b8abbca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6801","nonce":802398196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6803","nonce":802398196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6809","nonce":802398196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":802398196},{"type":"v1","addr":"172.21.15.169:6805","nonce":802398196}]},"public_addr":"172.21.15.169:6801/802398196","cluster_addr":"172.21.15.169:6803/802398196","heartbeat_back_addr":"172.21.15.169:6809/802398196","heartbeat_front_addr":"172.21.15.169:6805/802398196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:27.884755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.352528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.289531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.086498+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.191862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.063862+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.540264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:03:28.045534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:05:08.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 msgr2=0x7fcf900687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 0x7fcf900687a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00090b0 tx=0x7fcfa0010000 comp rx=0 tx=0).stop 2023-06-23T18:05:08.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 msgr2=0x7fcfac0b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800c520 tx=0x7fcf9800c830 comp rx=0 tx=0).stop 2023-06-23T18:05:08.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 shutdown_connections 2023-06-23T18:05:08.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfac159c30 0x7fcfac0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fcf900662f0 0x7fcf900687a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfac0affd0 0x7fcfac0b3480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 --2- 172.21.15.22:0/2991431578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfac071450 0x7fcfac155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 >> 172.21.15.22:0/2991431578 conn(0x7fcfac06a5f0 msgr2=0x7fcfac0cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 shutdown_connections 2023-06-23T18:05:08.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.059+0000 7fcfb1970700 1 -- 172.21.15.22:0/2991431578 wait complete. 2023-06-23T18:05:08.087 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-23T18:05:08.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:05:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.195+0000 7f68f886b700 1 Processor -- start 2023-06-23T18:05:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.195+0000 7f68f886b700 1 -- start start 2023-06-23T18:05:08.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f4070280 0x7f68f40afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f68f40affd0 0x7f68f40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68f40c93a0 0x7f68f40cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f41590a0 con 0x7f68f40affd0 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40b0420 con 0x7f68f40c93a0 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40c9b50 con 0x7f68f4070280 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f68f40affd0 0x7f68f40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f68f40affd0 0x7f68f40c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44544/0 (socket says 172.21.15.22:44544) 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 -- 172.21.15.22:0/3057998944 learned_addr learned my addr 172.21.15.22:0/3057998944 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f359e700 1 --2- 172.21.15.22:0/3057998944 >> v2:172.21.15.169:3300/0 conn(0x7f68f40c93a0 0x7f68f40cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f2d9d700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f4070280 0x7f68f40afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 -- 172.21.15.22:0/3057998944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f4070280 msgr2=0x7f68f40afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f4070280 0x7f68f40afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 -- 172.21.15.22:0/3057998944 >> v2:172.21.15.169:3300/0 conn(0x7f68f40c93a0 msgr2=0x7f68f40cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 --2- 172.21.15.22:0/3057998944 >> v2:172.21.15.169:3300/0 conn(0x7f68f40c93a0 0x7f68f40cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 -- 172.21.15.22:0/3057998944 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40c9e30 con 0x7f68f40affd0 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f259c700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f40affd0 0x7f68f40c8e60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f68f406d0b0 tx=0x7f68e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e4188c2c827a6a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f1d9b700 1 -- 172.21.15.22:0/3057998944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8011830 con 0x7f68f40affd0 2023-06-23T18:05:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f1d9b700 1 -- 172.21.15.22:0/3057998944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e8011990 con 0x7f68f40affd0 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f1d9b700 1 -- 172.21.15.22:0/3057998944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8011d20 con 0x7f68f40affd0 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- 172.21.15.22:0/3057998944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f40affd0 msgr2=0x7f68f40c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f40affd0 0x7f68f40c8e60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f68f406d0b0 tx=0x7f68e8009bf0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- 172.21.15.22:0/3057998944 shutdown_connections 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- 172.21.15.22:0/3057998944 >> v2:172.21.15.169:3300/0 conn(0x7f68f40c93a0 0x7f68f40cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f40affd0 0x7f68f40c8e60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 --2- 172.21.15.22:0/3057998944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f4070280 0x7f68f40afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- 172.21.15.22:0/3057998944 >> 172.21.15.22:0/3057998944 conn(0x7f68f406a5f0 msgr2=0x7f68f40caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- 172.21.15.22:0/3057998944 shutdown_connections 2023-06-23T18:05:08.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.199+0000 7f68f886b700 1 -- 172.21.15.22:0/3057998944 wait complete. 2023-06-23T18:05:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 Processor -- start 2023-06-23T18:05:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 -- start start 2023-06-23T18:05:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40c93a0 0x7f68f40c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f414f9f0 0x7f68f41525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40b0e40 con 0x7f68f4070280 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40c2f90 con 0x7f68f40c93a0 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f886b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40b3c80 con 0x7f68f414f9f0 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44556/0 (socket says 172.21.15.22:44556) 2023-06-23T18:05:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 -- 172.21.15.22:0/41041717 learned_addr learned my addr 172.21.15.22:0/41041717 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f359e700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f414f9f0 0x7f68f41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f259c700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40c93a0 0x7f68f40c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 -- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f414f9f0 msgr2=0x7f68f41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f414f9f0 0x7f68f41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 -- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40c93a0 msgr2=0x7f68f40c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40c93a0 0x7f68f40c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.203+0000 7f68f2d9d700 1 -- 172.21.15.22:0/41041717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8009570 con 0x7f68f4070280 2023-06-23T18:05:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68f2d9d700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f68e400ea00 tx=0x7f68e400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4009720 con 0x7f68f4070280 2023-06-23T18:05:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400eea0 con 0x7f68f4070280 2023-06-23T18:05:08.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f40c26f0 con 0x7f68f4070280 2023-06-23T18:05:08.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400ca50 con 0x7f68f4070280 2023-06-23T18:05:08.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f40c2cb0 con 0x7f68f4070280 2023-06-23T18:05:08.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f404e4c0 con 0x7f68f4070280 2023-06-23T18:05:08.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f68e400cbb0 con 0x7f68f4070280 2023-06-23T18:05:08.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.207+0000 7f68db7fe700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 0x7f68e00687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.211+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f68e4083e70 con 0x7f68f4070280 2023-06-23T18:05:08.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.211+0000 7f68f259c700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 0x7f68e00687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.211+0000 7f68f259c700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 0x7f68e00687d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68f40c1b10 tx=0x7f68e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.219+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f68e4054bc0 con 0x7f68f4070280 2023-06-23T18:05:08.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.383+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68f40ca3f0 con 0x7f68e0066320 2023-06-23T18:05:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.395+0000 7f68db7fe700 1 -- 172.21.15.22:0/41041717 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309868 (secure 0 0 0) 0x7f68e8011b00 con 0x7f68e0066320 2023-06-23T18:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 msgr2=0x7f68e00687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 0x7f68e00687d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68f40c1b10 tx=0x7f68e8008030 comp rx=0 tx=0).stop 2023-06-23T18:05:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 msgr2=0x7f68f40b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f68e400ea00 tx=0x7f68e400ed10 comp rx=0 tx=0).stop 2023-06-23T18:05:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 shutdown_connections 2023-06-23T18:05:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f414f9f0 0x7f68f41525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f68e0066320 0x7f68e00687d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40c93a0 0x7f68f40c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 --2- 172.21.15.22:0/41041717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68f4070280 0x7f68f40b3740 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 >> 172.21.15.22:0/41041717 conn(0x7f68f406a5f0 msgr2=0x7f68f406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 shutdown_connections 2023-06-23T18:05:08.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.403+0000 7f68f886b700 1 -- 172.21.15.22:0/41041717 wait complete. 2023-06-23T18:05:08.418 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-06-23T18:05:07.638722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236744,"kb_used_data":4744,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754737976,"statfs":{"total":773094113280,"available":772851687424,"internally_reserved":0,"allocated":4857856,"data_stored":2814236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942624"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441621+0000","last_change":"2023-06-23T18:05:04.120120+0000","last_active":"2023-06-23T18:05:05.441621+0000","last_peered":"2023-06-23T18:05:05.441621+0000","last_clean":"2023-06-23T18:05:05.441621+0000","last_became_active":"2023-06-23T18:05:04.119656+0000","last_became_peered":"2023-06-23T18:05:04.119656+0000","last_unstale":"2023-06-23T18:05:05.441621+0000","last_undegraded":"2023-06-23T18:05:05.441621+0000","last_fullsized":"2023-06-23T18:05:05.441621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:56.303413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440417+0000","last_change":"2023-06-23T18:05:04.136281+0000","last_active":"2023-06-23T18:05:05.440417+0000","last_peered":"2023-06-23T18:05:05.440417+0000","last_clean":"2023-06-23T18:05:05.440417+0000","last_became_active":"2023-06-23T18:05:04.136011+0000","last_became_peered":"2023-06-23T18:05:04.136011+0000","last_unstale":"2023-06-23T18:05:05.440417+0000","last_undegraded":"2023-06-23T18:05:05.440417+0000","last_fullsized":"2023-06-23T18:05:05.440417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:21.426316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113668+0000","last_change":"2023-06-23T18:05:04.119079+0000","last_active":"2023-06-23T18:05:05.113668+0000","last_peered":"2023-06-23T18:05:05.113668+0000","last_clean":"2023-06-23T18:05:05.113668+0000","last_became_active":"2023-06-23T18:05:04.118752+0000","last_became_peered":"2023-06-23T18:05:04.118752+0000","last_unstale":"2023-06-23T18:05:05.113668+0000","last_undegraded":"2023-06-23T18:05:05.113668+0000","last_fullsized":"2023-06-23T18:05:05.113668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:51:11.597188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440044+0000","last_change":"2023-06-23T18:05:04.135013+0000","last_active":"2023-06-23T18:05:05.440044+0000","last_peered":"2023-06-23T18:05:05.440044+0000","last_clean":"2023-06-23T18:05:05.440044+0000","last_became_active":"2023-06-23T18:05:04.134744+0000","last_became_peered":"2023-06-23T18:05:04.134744+0000","last_unstale":"2023-06-23T18:05:05.440044+0000","last_undegraded":"2023-06-23T18:05:05.440044+0000","last_fullsized":"2023-06-23T18:05:05.440044+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:54.419525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439278+0000","last_change":"2023-06-23T18:05:04.136379+0000","last_active":"2023-06-23T18:05:05.439278+0000","last_peered":"2023-06-23T18:05:05.439278+0000","last_clean":"2023-06-23T18:05:05.439278+0000","last_became_active":"2023-06-23T18:05:04.136050+0000","last_became_peered":"2023-06-23T18:05:04.136050+0000","last_unstale":"2023-06-23T18:05:05.439278+0000","last_undegraded":"2023-06-23T18:05:05.439278+0000","last_fullsized":"2023-06-23T18:05:05.439278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:26.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110207+0000","last_change":"2023-06-23T18:05:04.124092+0000","last_active":"2023-06-23T18:05:05.110207+0000","last_peered":"2023-06-23T18:05:05.110207+0000","last_clean":"2023-06-23T18:05:05.110207+0000","last_became_active":"2023-06-23T18:05:04.123481+0000","last_became_peered":"2023-06-23T18:05:04.123481+0000","last_unstale":"2023-06-23T18:05:05.110207+0000","last_undegraded":"2023-06-23T18:05:05.110207+0000","last_fullsized":"2023-06-23T18:05:05.110207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:17:03.193671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441539+0000","last_change":"2023-06-23T18:05:04.124289+0000","last_active":"2023-06-23T18:05:05.441539+0000","last_peered":"2023-06-23T18:05:05.441539+0000","last_clean":"2023-06-23T18:05:05.441539+0000","last_became_active":"2023-06-23T18:05:04.124058+0000","last_became_peered":"2023-06-23T18:05:04.124058+0000","last_unstale":"2023-06-23T18:05:05.441539+0000","last_undegraded":"2023-06-23T18:05:05.441539+0000","last_fullsized":"2023-06-23T18:05:05.441539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440130+0000","last_change":"2023-06-23T18:05:04.135114+0000","last_active":"2023-06-23T18:05:05.440130+0000","last_peered":"2023-06-23T18:05:05.440130+0000","last_clean":"2023-06-23T18:05:05.440130+0000","last_became_active":"2023-06-23T18:05:04.134739+0000","last_became_peered":"2023-06-23T18:05:04.134739+0000","last_unstale":"2023-06-23T18:05:05.440130+0000","last_undegraded":"2023-06-23T18:05:05.440130+0000","last_fullsized":"2023-06-23T18:05:05.440130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:14.980285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438666+0000","last_change":"2023-06-23T18:05:04.134531+0000","last_active":"2023-06-23T18:05:05.438666+0000","last_peered":"2023-06-23T18:05:05.438666+0000","last_clean":"2023-06-23T18:05:05.438666+0000","last_became_active":"2023-06-23T18:05:04.134220+0000","last_became_peered":"2023-06-23T18:05:04.134220+0000","last_unstale":"2023-06-23T18:05:05.438666+0000","last_undegraded":"2023-06-23T18:05:05.438666+0000","last_fullsized":"2023-06-23T18:05:05.438666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:23:43.253102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441086+0000","last_change":"2023-06-23T18:05:04.130561+0000","last_active":"2023-06-23T18:05:05.441086+0000","last_peered":"2023-06-23T18:05:05.441086+0000","last_clean":"2023-06-23T18:05:05.441086+0000","last_became_active":"2023-06-23T18:05:04.130303+0000","last_became_peered":"2023-06-23T18:05:04.130303+0000","last_unstale":"2023-06-23T18:05:05.441086+0000","last_undegraded":"2023-06-23T18:05:05.441086+0000","last_fullsized":"2023-06-23T18:05:05.441086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439913+0000","last_change":"2023-06-23T18:05:04.124947+0000","last_active":"2023-06-23T18:05:05.439913+0000","last_peered":"2023-06-23T18:05:05.439913+0000","last_clean":"2023-06-23T18:05:05.439913+0000","last_became_active":"2023-06-23T18:05:04.124698+0000","last_became_peered":"2023-06-23T18:05:04.124698+0000","last_unstale":"2023-06-23T18:05:05.439913+0000","last_undegraded":"2023-06-23T18:05:05.439913+0000","last_fullsized":"2023-06-23T18:05:05.439913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:15.534113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439984+0000","last_change":"2023-06-23T18:05:04.135170+0000","last_active":"2023-06-23T18:05:05.439984+0000","last_peered":"2023-06-23T18:05:05.439984+0000","last_clean":"2023-06-23T18:05:05.439984+0000","last_became_active":"2023-06-23T18:05:04.134897+0000","last_became_peered":"2023-06-23T18:05:04.134897+0000","last_unstale":"2023-06-23T18:05:05.439984+0000","last_undegraded":"2023-06-23T18:05:05.439984+0000","last_fullsized":"2023-06-23T18:05:05.439984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438109+0000","last_change":"2023-06-23T18:05:04.124468+0000","last_active":"2023-06-23T18:05:05.438109+0000","last_peered":"2023-06-23T18:05:05.438109+0000","last_clean":"2023-06-23T18:05:05.438109+0000","last_became_active":"2023-06-23T18:05:04.124222+0000","last_became_peered":"2023-06-23T18:05:04.124222+0000","last_unstale":"2023-06-23T18:05:05.438109+0000","last_undegraded":"2023-06-23T18:05:05.438109+0000","last_fullsized":"2023-06-23T18:05:05.438109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:09:59.421209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439671+0000","last_change":"2023-06-23T18:05:04.119644+0000","last_active":"2023-06-23T18:05:05.439671+0000","last_peered":"2023-06-23T18:05:05.439671+0000","last_clean":"2023-06-23T18:05:05.439671+0000","last_became_active":"2023-06-23T18:05:04.119384+0000","last_became_peered":"2023-06-23T18:05:04.119384+0000","last_unstale":"2023-06-23T18:05:05.439671+0000","last_undegraded":"2023-06-23T18:05:05.439671+0000","last_fullsized":"2023-06-23T18:05:05.439671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:27:28.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.109772+0000","last_change":"2023-06-23T18:05:04.123945+0000","last_active":"2023-06-23T18:05:05.109772+0000","last_peered":"2023-06-23T18:05:05.109772+0000","last_clean":"2023-06-23T18:05:05.109772+0000","last_became_active":"2023-06-23T18:05:04.123620+0000","last_became_peered":"2023-06-23T18:05:04.123620+0000","last_unstale":"2023-06-23T18:05:05.109772+0000","last_undegraded":"2023-06-23T18:05:05.109772+0000","last_fullsized":"2023-06-23T18:05:05.109772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:31.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439710+0000","last_change":"2023-06-23T18:05:04.136041+0000","last_active":"2023-06-23T18:05:05.439710+0000","last_peered":"2023-06-23T18:05:05.439710+0000","last_clean":"2023-06-23T18:05:05.439710+0000","last_became_active":"2023-06-23T18:05:04.135746+0000","last_became_peered":"2023-06-23T18:05:04.135746+0000","last_unstale":"2023-06-23T18:05:05.439710+0000","last_undegraded":"2023-06-23T18:05:05.439710+0000","last_fullsized":"2023-06-23T18:05:05.439710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:29.436604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113343+0000","last_change":"2023-06-23T18:05:04.123364+0000","last_active":"2023-06-23T18:05:05.113343+0000","last_peered":"2023-06-23T18:05:05.113343+0000","last_clean":"2023-06-23T18:05:05.113343+0000","last_became_active":"2023-06-23T18:05:04.123005+0000","last_became_peered":"2023-06-23T18:05:04.123005+0000","last_unstale":"2023-06-23T18:05:05.113343+0000","last_undegraded":"2023-06-23T18:05:05.113343+0000","last_fullsized":"2023-06-23T18:05:05.113343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:09:27.091323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113136+0000","last_change":"2023-06-23T18:05:04.136249+0000","last_active":"2023-06-23T18:05:05.113136+0000","last_peered":"2023-06-23T18:05:05.113136+0000","last_clean":"2023-06-23T18:05:05.113136+0000","last_became_active":"2023-06-23T18:05:04.135943+0000","last_became_peered":"2023-06-23T18:05:04.135943+0000","last_unstale":"2023-06-23T18:05:05.113136+0000","last_undegraded":"2023-06-23T18:05:05.113136+0000","last_fullsized":"2023-06-23T18:05:05.113136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:02.966525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439140+0000","last_change":"2023-06-23T18:05:04.136717+0000","last_active":"2023-06-23T18:05:05.439140+0000","last_peered":"2023-06-23T18:05:05.439140+0000","last_clean":"2023-06-23T18:05:05.439140+0000","last_became_active":"2023-06-23T18:05:04.136439+0000","last_became_peered":"2023-06-23T18:05:04.136439+0000","last_unstale":"2023-06-23T18:05:05.439140+0000","last_undegraded":"2023-06-23T18:05:05.439140+0000","last_fullsized":"2023-06-23T18:05:05.439140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:05:22.128448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439200+0000","last_change":"2023-06-23T18:05:04.123307+0000","last_active":"2023-06-23T18:05:05.439200+0000","last_peered":"2023-06-23T18:05:05.439200+0000","last_clean":"2023-06-23T18:05:05.439200+0000","last_became_active":"2023-06-23T18:05:04.123100+0000","last_became_peered":"2023-06-23T18:05:04.123100+0000","last_unstale":"2023-06-23T18:05:05.439200+0000","last_undegraded":"2023-06-23T18:05:05.439200+0000","last_fullsized":"2023-06-23T18:05:05.439200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:46:43.742354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.109990+0000","last_change":"2023-06-23T18:05:04.131916+0000","last_active":"2023-06-23T18:05:05.109990+0000","last_peered":"2023-06-23T18:05:05.109990+0000","last_clean":"2023-06-23T18:05:05.109990+0000","last_became_active":"2023-06-23T18:05:04.131334+0000","last_became_peered":"2023-06-23T18:05:04.131334+0000","last_unstale":"2023-06-23T18:05:05.109990+0000","last_undegraded":"2023-06-23T18:05:05.109990+0000","last_fullsized":"2023-06-23T18:05:05.109990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:26:00.815297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439652+0000","last_change":"2023-06-23T18:05:04.130365+0000","last_active":"2023-06-23T18:05:05.439652+0000","last_peered":"2023-06-23T18:05:05.439652+0000","last_clean":"2023-06-23T18:05:05.439652+0000","last_became_active":"2023-06-23T18:05:04.130012+0000","last_became_peered":"2023-06-23T18:05:04.130012+0000","last_unstale":"2023-06-23T18:05:05.439652+0000","last_undegraded":"2023-06-23T18:05:05.439652+0000","last_fullsized":"2023-06-23T18:05:05.439652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:45:24.916678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438862+0000","last_change":"2023-06-23T18:05:04.124423+0000","last_active":"2023-06-23T18:05:05.438862+0000","last_peered":"2023-06-23T18:05:05.438862+0000","last_clean":"2023-06-23T18:05:05.438862+0000","last_became_active":"2023-06-23T18:05:04.124106+0000","last_became_peered":"2023-06-23T18:05:04.124106+0000","last_unstale":"2023-06-23T18:05:05.438862+0000","last_undegraded":"2023-06-23T18:05:05.438862+0000","last_fullsized":"2023-06-23T18:05:05.438862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:28.379296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438694+0000","last_change":"2023-06-23T18:05:04.123494+0000","last_active":"2023-06-23T18:05:05.438694+0000","last_peered":"2023-06-23T18:05:05.438694+0000","last_clean":"2023-06-23T18:05:05.438694+0000","last_became_active":"2023-06-23T18:05:04.123196+0000","last_became_peered":"2023-06-23T18:05:04.123196+0000","last_unstale":"2023-06-23T18:05:05.438694+0000","last_undegraded":"2023-06-23T18:05:05.438694+0000","last_fullsized":"2023-06-23T18:05:05.438694+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:29:03.099380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439297+0000","last_change":"2023-06-23T18:05:04.132615+0000","last_active":"2023-06-23T18:05:05.439297+0000","last_peered":"2023-06-23T18:05:05.439297+0000","last_clean":"2023-06-23T18:05:05.439297+0000","last_became_active":"2023-06-23T18:05:04.130611+0000","last_became_peered":"2023-06-23T18:05:04.130611+0000","last_unstale":"2023-06-23T18:05:05.439297+0000","last_undegraded":"2023-06-23T18:05:05.439297+0000","last_fullsized":"2023-06-23T18:05:05.439297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:55:39.715711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438094+0000","last_change":"2023-06-23T18:05:04.123816+0000","last_active":"2023-06-23T18:05:05.438094+0000","last_peered":"2023-06-23T18:05:05.438094+0000","last_clean":"2023-06-23T18:05:05.438094+0000","last_became_active":"2023-06-23T18:05:04.123570+0000","last_became_peered":"2023-06-23T18:05:04.123570+0000","last_unstale":"2023-06-23T18:05:05.438094+0000","last_undegraded":"2023-06-23T18:05:05.438094+0000","last_fullsized":"2023-06-23T18:05:05.438094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:47.889106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113850+0000","last_change":"2023-06-23T18:05:04.123820+0000","last_active":"2023-06-23T18:05:05.113850+0000","last_peered":"2023-06-23T18:05:05.113850+0000","last_clean":"2023-06-23T18:05:05.113850+0000","last_became_active":"2023-06-23T18:05:04.123508+0000","last_became_peered":"2023-06-23T18:05:04.123508+0000","last_unstale":"2023-06-23T18:05:05.113850+0000","last_undegraded":"2023-06-23T18:05:05.113850+0000","last_fullsized":"2023-06-23T18:05:05.113850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:41:51.703626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440184+0000","last_change":"2023-06-23T18:05:04.135566+0000","last_active":"2023-06-23T18:05:05.440184+0000","last_peered":"2023-06-23T18:05:05.440184+0000","last_clean":"2023-06-23T18:05:05.440184+0000","last_became_active":"2023-06-23T18:05:04.135286+0000","last_became_peered":"2023-06-23T18:05:04.135286+0000","last_unstale":"2023-06-23T18:05:05.440184+0000","last_undegraded":"2023-06-23T18:05:05.440184+0000","last_fullsized":"2023-06-23T18:05:05.440184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439356+0000","last_change":"2023-06-23T18:05:04.124948+0000","last_active":"2023-06-23T18:05:05.439356+0000","last_peered":"2023-06-23T18:05:05.439356+0000","last_clean":"2023-06-23T18:05:05.439356+0000","last_became_active":"2023-06-23T18:05:04.124675+0000","last_became_peered":"2023-06-23T18:05:04.124675+0000","last_unstale":"2023-06-23T18:05:05.439356+0000","last_undegraded":"2023-06-23T18:05:05.439356+0000","last_fullsized":"2023-06-23T18:05:05.439356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:31.217898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.112867+0000","last_change":"2023-06-23T18:05:04.120917+0000","last_active":"2023-06-23T18:05:05.112867+0000","last_peered":"2023-06-23T18:05:05.112867+0000","last_clean":"2023-06-23T18:05:05.112867+0000","last_became_active":"2023-06-23T18:05:04.120639+0000","last_became_peered":"2023-06-23T18:05:04.120639+0000","last_unstale":"2023-06-23T18:05:05.112867+0000","last_undegraded":"2023-06-23T18:05:05.112867+0000","last_fullsized":"2023-06-23T18:05:05.112867+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:53:32.131393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110032+0000","last_change":"2023-06-23T18:05:04.123746+0000","last_active":"2023-06-23T18:05:05.110032+0000","last_peered":"2023-06-23T18:05:05.110032+0000","last_clean":"2023-06-23T18:05:05.110032+0000","last_became_active":"2023-06-23T18:05:04.123436+0000","last_became_peered":"2023-06-23T18:05:04.123436+0000","last_unstale":"2023-06-23T18:05:05.110032+0000","last_undegraded":"2023-06-23T18:05:05.110032+0000","last_fullsized":"2023-06-23T18:05:05.110032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:41:15.938263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437424+0000","last_change":"2023-06-23T18:05:04.114783+0000","last_active":"2023-06-23T18:05:05.437424+0000","last_peered":"2023-06-23T18:05:05.437424+0000","last_clean":"2023-06-23T18:05:05.437424+0000","last_became_active":"2023-06-23T18:05:04.114523+0000","last_became_peered":"2023-06-23T18:05:04.114523+0000","last_unstale":"2023-06-23T18:05:05.437424+0000","last_undegraded":"2023-06-23T18:05:05.437424+0000","last_fullsized":"2023-06-23T18:05:05.437424+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:02:36.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438848+0000","last_change":"2023-06-23T18:05:04.123924+0000","last_active":"2023-06-23T18:05:05.438848+0000","last_peered":"2023-06-23T18:05:05.438848+0000","last_clean":"2023-06-23T18:05:05.438848+0000","last_became_active":"2023-06-23T18:05:04.123682+0000","last_became_peered":"2023-06-23T18:05:04.123682+0000","last_unstale":"2023-06-23T18:05:05.438848+0000","last_undegraded":"2023-06-23T18:05:05.438848+0000","last_fullsized":"2023-06-23T18:05:05.438848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:31.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110526+0000","last_change":"2023-06-23T18:05:03.728918+0000","last_active":"2023-06-23T18:05:05.110526+0000","last_peered":"2023-06-23T18:05:05.110526+0000","last_clean":"2023-06-23T18:05:05.110526+0000","last_became_active":"2023-06-23T18:05:03.728637+0000","last_became_peered":"2023-06-23T18:05:03.728637+0000","last_unstale":"2023-06-23T18:05:05.110526+0000","last_undegraded":"2023-06-23T18:05:05.110526+0000","last_fullsized":"2023-06-23T18:05:05.110526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:41.096078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439479+0000","last_change":"2023-06-23T18:05:04.131916+0000","last_active":"2023-06-23T18:05:05.439479+0000","last_peered":"2023-06-23T18:05:05.439479+0000","last_clean":"2023-06-23T18:05:05.439479+0000","last_became_active":"2023-06-23T18:05:04.131556+0000","last_became_peered":"2023-06-23T18:05:04.131556+0000","last_unstale":"2023-06-23T18:05:05.439479+0000","last_undegraded":"2023-06-23T18:05:05.439479+0000","last_fullsized":"2023-06-23T18:05:05.439479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:51.742598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113448+0000","last_change":"2023-06-23T18:05:03.723617+0000","last_active":"2023-06-23T18:05:05.113448+0000","last_peered":"2023-06-23T18:05:05.113448+0000","last_clean":"2023-06-23T18:05:05.113448+0000","last_became_active":"2023-06-23T18:05:03.723356+0000","last_became_peered":"2023-06-23T18:05:03.723356+0000","last_unstale":"2023-06-23T18:05:05.113448+0000","last_undegraded":"2023-06-23T18:05:05.113448+0000","last_fullsized":"2023-06-23T18:05:05.113448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:15:36.304210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440219+0000","last_change":"2023-06-23T18:05:04.128971+0000","last_active":"2023-06-23T18:05:05.440219+0000","last_peered":"2023-06-23T18:05:05.440219+0000","last_clean":"2023-06-23T18:05:05.440219+0000","last_became_active":"2023-06-23T18:05:04.128263+0000","last_became_peered":"2023-06-23T18:05:04.128263+0000","last_unstale":"2023-06-23T18:05:05.440219+0000","last_undegraded":"2023-06-23T18:05:05.440219+0000","last_fullsized":"2023-06-23T18:05:05.440219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:56.303413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439160+0000","last_change":"2023-06-23T18:05:03.724885+0000","last_active":"2023-06-23T18:05:05.439160+0000","last_peered":"2023-06-23T18:05:05.439160+0000","last_clean":"2023-06-23T18:05:05.439160+0000","last_became_active":"2023-06-23T18:05:03.724514+0000","last_became_peered":"2023-06-23T18:05:03.724514+0000","last_unstale":"2023-06-23T18:05:05.439160+0000","last_undegraded":"2023-06-23T18:05:05.439160+0000","last_fullsized":"2023-06-23T18:05:05.439160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:59.811433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438814+0000","last_change":"2023-06-23T18:05:04.130399+0000","last_active":"2023-06-23T18:05:05.438814+0000","last_peered":"2023-06-23T18:05:05.438814+0000","last_clean":"2023-06-23T18:05:05.438814+0000","last_became_active":"2023-06-23T18:05:04.130108+0000","last_became_peered":"2023-06-23T18:05:04.130108+0000","last_unstale":"2023-06-23T18:05:05.438814+0000","last_undegraded":"2023-06-23T18:05:05.438814+0000","last_fullsized":"2023-06-23T18:05:05.438814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:36:14.870964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441514+0000","last_change":"2023-06-23T18:05:03.722805+0000","last_active":"2023-06-23T18:05:05.441514+0000","last_peered":"2023-06-23T18:05:05.441514+0000","last_clean":"2023-06-23T18:05:05.441514+0000","last_became_active":"2023-06-23T18:05:03.722448+0000","last_became_peered":"2023-06-23T18:05:03.722448+0000","last_unstale":"2023-06-23T18:05:05.441514+0000","last_undegraded":"2023-06-23T18:05:05.441514+0000","last_fullsized":"2023-06-23T18:05:05.441514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:41.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113091+0000","last_change":"2023-06-23T18:05:04.131677+0000","last_active":"2023-06-23T18:05:05.113091+0000","last_peered":"2023-06-23T18:05:05.113091+0000","last_clean":"2023-06-23T18:05:05.113091+0000","last_became_active":"2023-06-23T18:05:04.131405+0000","last_became_peered":"2023-06-23T18:05:04.131405+0000","last_unstale":"2023-06-23T18:05:05.113091+0000","last_undegraded":"2023-06-23T18:05:05.113091+0000","last_fullsized":"2023-06-23T18:05:05.113091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:02:44.968541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110284+0000","last_change":"2023-06-23T18:05:03.727582+0000","last_active":"2023-06-23T18:05:05.110284+0000","last_peered":"2023-06-23T18:05:05.110284+0000","last_clean":"2023-06-23T18:05:05.110284+0000","last_became_active":"2023-06-23T18:05:03.727119+0000","last_became_peered":"2023-06-23T18:05:03.727119+0000","last_unstale":"2023-06-23T18:05:05.110284+0000","last_undegraded":"2023-06-23T18:05:05.110284+0000","last_fullsized":"2023-06-23T18:05:05.110284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:27:27.895179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438698+0000","last_change":"2023-06-23T18:05:04.134695+0000","last_active":"2023-06-23T18:05:05.438698+0000","last_peered":"2023-06-23T18:05:05.438698+0000","last_clean":"2023-06-23T18:05:05.438698+0000","last_became_active":"2023-06-23T18:05:04.134276+0000","last_became_peered":"2023-06-23T18:05:04.134276+0000","last_unstale":"2023-06-23T18:05:05.438698+0000","last_undegraded":"2023-06-23T18:05:05.438698+0000","last_fullsized":"2023-06-23T18:05:05.438698+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:52:35.317430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113090+0000","last_change":"2023-06-23T18:05:03.722113+0000","last_active":"2023-06-23T18:05:05.113090+0000","last_peered":"2023-06-23T18:05:05.113090+0000","last_clean":"2023-06-23T18:05:05.113090+0000","last_became_active":"2023-06-23T18:05:03.721844+0000","last_became_peered":"2023-06-23T18:05:03.721844+0000","last_unstale":"2023-06-23T18:05:05.113090+0000","last_undegraded":"2023-06-23T18:05:05.113090+0000","last_fullsized":"2023-06-23T18:05:05.113090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:51:14.835648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440408+0000","last_change":"2023-06-23T18:05:04.130219+0000","last_active":"2023-06-23T18:05:05.440408+0000","last_peered":"2023-06-23T18:05:05.440408+0000","last_clean":"2023-06-23T18:05:05.440408+0000","last_became_active":"2023-06-23T18:05:04.129947+0000","last_became_peered":"2023-06-23T18:05:04.129947+0000","last_unstale":"2023-06-23T18:05:05.440408+0000","last_undegraded":"2023-06-23T18:05:05.440408+0000","last_fullsized":"2023-06-23T18:05:05.440408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:52:17.794961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438542+0000","last_change":"2023-06-23T18:05:03.728235+0000","last_active":"2023-06-23T18:05:05.438542+0000","last_peered":"2023-06-23T18:05:05.438542+0000","last_clean":"2023-06-23T18:05:05.438542+0000","last_became_active":"2023-06-23T18:05:03.727919+0000","last_became_peered":"2023-06-23T18:05:03.727919+0000","last_unstale":"2023-06-23T18:05:05.438542+0000","last_undegraded":"2023-06-23T18:05:05.438542+0000","last_fullsized":"2023-06-23T18:05:05.438542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:13.976421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439549+0000","last_change":"2023-06-23T18:05:04.124417+0000","last_active":"2023-06-23T18:05:05.439549+0000","last_peered":"2023-06-23T18:05:05.439549+0000","last_clean":"2023-06-23T18:05:05.439549+0000","last_became_active":"2023-06-23T18:05:04.124109+0000","last_became_peered":"2023-06-23T18:05:04.124109+0000","last_unstale":"2023-06-23T18:05:05.439549+0000","last_undegraded":"2023-06-23T18:05:05.439549+0000","last_fullsized":"2023-06-23T18:05:05.439549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:06.951667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440368+0000","last_change":"2023-06-23T18:05:03.727412+0000","last_active":"2023-06-23T18:05:05.440368+0000","last_peered":"2023-06-23T18:05:05.440368+0000","last_clean":"2023-06-23T18:05:05.440368+0000","last_became_active":"2023-06-23T18:05:03.727120+0000","last_became_peered":"2023-06-23T18:05:03.727120+0000","last_unstale":"2023-06-23T18:05:05.440368+0000","last_undegraded":"2023-06-23T18:05:05.440368+0000","last_fullsized":"2023-06-23T18:05:05.440368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113374+0000","last_change":"2023-06-23T18:05:04.137057+0000","last_active":"2023-06-23T18:05:05.113374+0000","last_peered":"2023-06-23T18:05:05.113374+0000","last_clean":"2023-06-23T18:05:05.113374+0000","last_became_active":"2023-06-23T18:05:04.136732+0000","last_became_peered":"2023-06-23T18:05:04.136732+0000","last_unstale":"2023-06-23T18:05:05.113374+0000","last_undegraded":"2023-06-23T18:05:05.113374+0000","last_fullsized":"2023-06-23T18:05:05.113374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:16:11.793941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113589+0000","last_change":"2023-06-23T18:05:03.729653+0000","last_active":"2023-06-23T18:05:05.113589+0000","last_peered":"2023-06-23T18:05:05.113589+0000","last_clean":"2023-06-23T18:05:05.113589+0000","last_became_active":"2023-06-23T18:05:03.729385+0000","last_became_peered":"2023-06-23T18:05:03.729385+0000","last_unstale":"2023-06-23T18:05:05.113589+0000","last_undegraded":"2023-06-23T18:05:05.113589+0000","last_fullsized":"2023-06-23T18:05:05.113589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:52:31.861434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441634+0000","last_change":"2023-06-23T18:05:04.123510+0000","last_active":"2023-06-23T18:05:05.441634+0000","last_peered":"2023-06-23T18:05:05.441634+0000","last_clean":"2023-06-23T18:05:05.441634+0000","last_became_active":"2023-06-23T18:05:04.123234+0000","last_became_peered":"2023-06-23T18:05:04.123234+0000","last_unstale":"2023-06-23T18:05:05.441634+0000","last_undegraded":"2023-06-23T18:05:05.441634+0000","last_fullsized":"2023-06-23T18:05:05.441634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:26.466042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441391+0000","last_change":"2023-06-23T18:05:03.728901+0000","last_active":"2023-06-23T18:05:05.441391+0000","last_peered":"2023-06-23T18:05:05.441391+0000","last_clean":"2023-06-23T18:05:05.441391+0000","last_became_active":"2023-06-23T18:05:03.728625+0000","last_became_peered":"2023-06-23T18:05:03.728625+0000","last_unstale":"2023-06-23T18:05:05.441391+0000","last_undegraded":"2023-06-23T18:05:05.441391+0000","last_fullsized":"2023-06-23T18:05:05.441391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:55.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110220+0000","last_change":"2023-06-23T18:05:04.126873+0000","last_active":"2023-06-23T18:05:05.110220+0000","last_peered":"2023-06-23T18:05:05.110220+0000","last_clean":"2023-06-23T18:05:05.110220+0000","last_became_active":"2023-06-23T18:05:04.126607+0000","last_became_peered":"2023-06-23T18:05:04.126607+0000","last_unstale":"2023-06-23T18:05:05.110220+0000","last_undegraded":"2023-06-23T18:05:05.110220+0000","last_fullsized":"2023-06-23T18:05:05.110220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:12:29.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441738+0000","last_change":"2023-06-23T18:05:03.725386+0000","last_active":"2023-06-23T18:05:05.441738+0000","last_peered":"2023-06-23T18:05:05.441738+0000","last_clean":"2023-06-23T18:05:05.441738+0000","last_became_active":"2023-06-23T18:05:03.725139+0000","last_became_peered":"2023-06-23T18:05:03.725139+0000","last_unstale":"2023-06-23T18:05:05.441738+0000","last_undegraded":"2023-06-23T18:05:05.441738+0000","last_fullsized":"2023-06-23T18:05:05.441738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:20.422452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441397+0000","last_change":"2023-06-23T18:05:04.126145+0000","last_active":"2023-06-23T18:05:05.441397+0000","last_peered":"2023-06-23T18:05:05.441397+0000","last_clean":"2023-06-23T18:05:05.441397+0000","last_became_active":"2023-06-23T18:05:04.125823+0000","last_became_peered":"2023-06-23T18:05:04.125823+0000","last_unstale":"2023-06-23T18:05:05.441397+0000","last_undegraded":"2023-06-23T18:05:05.441397+0000","last_fullsized":"2023-06-23T18:05:05.441397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:26.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439522+0000","last_change":"2023-06-23T18:05:03.721650+0000","last_active":"2023-06-23T18:05:05.439522+0000","last_peered":"2023-06-23T18:05:05.439522+0000","last_clean":"2023-06-23T18:05:05.439522+0000","last_became_active":"2023-06-23T18:05:03.721298+0000","last_became_peered":"2023-06-23T18:05:03.721298+0000","last_unstale":"2023-06-23T18:05:05.439522+0000","last_undegraded":"2023-06-23T18:05:05.439522+0000","last_fullsized":"2023-06-23T18:05:05.439522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:48:43.835290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439677+0000","last_change":"2023-06-23T18:05:04.129562+0000","last_active":"2023-06-23T18:05:05.439677+0000","last_peered":"2023-06-23T18:05:05.439677+0000","last_clean":"2023-06-23T18:05:05.439677+0000","last_became_active":"2023-06-23T18:05:04.129260+0000","last_became_peered":"2023-06-23T18:05:04.129260+0000","last_unstale":"2023-06-23T18:05:05.439677+0000","last_undegraded":"2023-06-23T18:05:05.439677+0000","last_fullsized":"2023-06-23T18:05:05.439677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:58:08.117888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110644+0000","last_change":"2023-06-23T18:05:03.729508+0000","last_active":"2023-06-23T18:05:05.110644+0000","last_peered":"2023-06-23T18:05:05.110644+0000","last_clean":"2023-06-23T18:05:05.110644+0000","last_became_active":"2023-06-23T18:05:03.729232+0000","last_became_peered":"2023-06-23T18:05:03.729232+0000","last_unstale":"2023-06-23T18:05:05.110644+0000","last_undegraded":"2023-06-23T18:05:05.110644+0000","last_fullsized":"2023-06-23T18:05:05.110644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439006+0000","last_change":"2023-06-23T18:05:04.119501+0000","last_active":"2023-06-23T18:05:05.439006+0000","last_peered":"2023-06-23T18:05:05.439006+0000","last_clean":"2023-06-23T18:05:05.439006+0000","last_became_active":"2023-06-23T18:05:04.119253+0000","last_became_peered":"2023-06-23T18:05:04.119253+0000","last_unstale":"2023-06-23T18:05:05.439006+0000","last_undegraded":"2023-06-23T18:05:05.439006+0000","last_fullsized":"2023-06-23T18:05:05.439006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:12:29.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441769+0000","last_change":"2023-06-23T18:05:03.729828+0000","last_active":"2023-06-23T18:05:05.441769+0000","last_peered":"2023-06-23T18:05:05.441769+0000","last_clean":"2023-06-23T18:05:05.441769+0000","last_became_active":"2023-06-23T18:05:03.729456+0000","last_became_peered":"2023-06-23T18:05:03.729456+0000","last_unstale":"2023-06-23T18:05:05.441769+0000","last_undegraded":"2023-06-23T18:05:05.441769+0000","last_fullsized":"2023-06-23T18:05:05.441769+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:08.542108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110021+0000","last_change":"2023-06-23T18:05:04.126754+0000","last_active":"2023-06-23T18:05:05.110021+0000","last_peered":"2023-06-23T18:05:05.110021+0000","last_clean":"2023-06-23T18:05:05.110021+0000","last_became_active":"2023-06-23T18:05:04.126409+0000","last_became_peered":"2023-06-23T18:05:04.126409+0000","last_unstale":"2023-06-23T18:05:05.110021+0000","last_undegraded":"2023-06-23T18:05:05.110021+0000","last_fullsized":"2023-06-23T18:05:05.110021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:18:35.359050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110004+0000","last_change":"2023-06-23T18:05:03.725690+0000","last_active":"2023-06-23T18:05:05.110004+0000","last_peered":"2023-06-23T18:05:05.110004+0000","last_clean":"2023-06-23T18:05:05.110004+0000","last_became_active":"2023-06-23T18:05:03.725416+0000","last_became_peered":"2023-06-23T18:05:03.725416+0000","last_unstale":"2023-06-23T18:05:05.110004+0000","last_undegraded":"2023-06-23T18:05:05.110004+0000","last_fullsized":"2023-06-23T18:05:05.110004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:39:17.933184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441932+0000","last_change":"2023-06-23T18:05:04.132003+0000","last_active":"2023-06-23T18:05:05.441932+0000","last_peered":"2023-06-23T18:05:05.441932+0000","last_clean":"2023-06-23T18:05:05.441932+0000","last_became_active":"2023-06-23T18:05:04.131694+0000","last_became_peered":"2023-06-23T18:05:04.131694+0000","last_unstale":"2023-06-23T18:05:05.441932+0000","last_undegraded":"2023-06-23T18:05:05.441932+0000","last_fullsized":"2023-06-23T18:05:05.441932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:26:05.299659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440521+0000","last_change":"2023-06-23T18:05:03.720382+0000","last_active":"2023-06-23T18:05:05.440521+0000","last_peered":"2023-06-23T18:05:05.440521+0000","last_clean":"2023-06-23T18:05:05.440521+0000","last_became_active":"2023-06-23T18:05:03.720090+0000","last_became_peered":"2023-06-23T18:05:03.720090+0000","last_unstale":"2023-06-23T18:05:05.440521+0000","last_undegraded":"2023-06-23T18:05:05.440521+0000","last_fullsized":"2023-06-23T18:05:05.440521+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:25.615986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438444+0000","last_change":"2023-06-23T18:05:04.121482+0000","last_active":"2023-06-23T18:05:05.438444+0000","last_peered":"2023-06-23T18:05:05.438444+0000","last_clean":"2023-06-23T18:05:05.438444+0000","last_became_active":"2023-06-23T18:05:04.121232+0000","last_became_peered":"2023-06-23T18:05:04.121232+0000","last_unstale":"2023-06-23T18:05:05.438444+0000","last_undegraded":"2023-06-23T18:05:05.438444+0000","last_fullsized":"2023-06-23T18:05:05.438444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:15.785497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438941+0000","last_change":"2023-06-23T18:05:03.725388+0000","last_active":"2023-06-23T18:05:05.438941+0000","last_peered":"2023-06-23T18:05:05.438941+0000","last_clean":"2023-06-23T18:05:05.438941+0000","last_became_active":"2023-06-23T18:05:03.725067+0000","last_became_peered":"2023-06-23T18:05:03.725067+0000","last_unstale":"2023-06-23T18:05:05.438941+0000","last_undegraded":"2023-06-23T18:05:05.438941+0000","last_fullsized":"2023-06-23T18:05:05.438941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:44:16.015011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439005+0000","last_change":"2023-06-23T18:05:04.127531+0000","last_active":"2023-06-23T18:05:05.439005+0000","last_peered":"2023-06-23T18:05:05.439005+0000","last_clean":"2023-06-23T18:05:05.439005+0000","last_became_active":"2023-06-23T18:05:04.127278+0000","last_became_peered":"2023-06-23T18:05:04.127278+0000","last_unstale":"2023-06-23T18:05:05.439005+0000","last_undegraded":"2023-06-23T18:05:05.439005+0000","last_fullsized":"2023-06-23T18:05:05.439005+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:17:03.193671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113539+0000","last_change":"2023-06-23T18:05:03.727766+0000","last_active":"2023-06-23T18:05:05.113539+0000","last_peered":"2023-06-23T18:05:05.113539+0000","last_clean":"2023-06-23T18:05:05.113539+0000","last_became_active":"2023-06-23T18:05:03.727121+0000","last_became_peered":"2023-06-23T18:05:03.727121+0000","last_unstale":"2023-06-23T18:05:05.113539+0000","last_undegraded":"2023-06-23T18:05:05.113539+0000","last_fullsized":"2023-06-23T18:05:05.113539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:52.582036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439224+0000","last_change":"2023-06-23T18:05:04.127883+0000","last_active":"2023-06-23T18:05:05.439224+0000","last_peered":"2023-06-23T18:05:05.439224+0000","last_clean":"2023-06-23T18:05:05.439224+0000","last_became_active":"2023-06-23T18:05:04.127597+0000","last_became_peered":"2023-06-23T18:05:04.127597+0000","last_unstale":"2023-06-23T18:05:05.439224+0000","last_undegraded":"2023-06-23T18:05:05.439224+0000","last_fullsized":"2023-06-23T18:05:05.439224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:44:26.837150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440302+0000","last_change":"2023-06-23T18:05:03.731975+0000","last_active":"2023-06-23T18:05:05.440302+0000","last_peered":"2023-06-23T18:05:05.440302+0000","last_clean":"2023-06-23T18:05:05.440302+0000","last_became_active":"2023-06-23T18:05:03.731696+0000","last_became_peered":"2023-06-23T18:05:03.731696+0000","last_unstale":"2023-06-23T18:05:05.440302+0000","last_undegraded":"2023-06-23T18:05:05.440302+0000","last_fullsized":"2023-06-23T18:05:05.440302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:25.462178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439527+0000","last_change":"2023-06-23T18:05:04.131040+0000","last_active":"2023-06-23T18:05:05.439527+0000","last_peered":"2023-06-23T18:05:05.439527+0000","last_clean":"2023-06-23T18:05:05.439527+0000","last_became_active":"2023-06-23T18:05:04.130585+0000","last_became_peered":"2023-06-23T18:05:04.130585+0000","last_unstale":"2023-06-23T18:05:05.439527+0000","last_undegraded":"2023-06-23T18:05:05.439527+0000","last_fullsized":"2023-06-23T18:05:05.439527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:53:12.390982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110265+0000","last_change":"2023-06-23T18:05:03.725776+0000","last_active":"2023-06-23T18:05:05.110265+0000","last_peered":"2023-06-23T18:05:05.110265+0000","last_clean":"2023-06-23T18:05:05.110265+0000","last_became_active":"2023-06-23T18:05:03.725254+0000","last_became_peered":"2023-06-23T18:05:03.725254+0000","last_unstale":"2023-06-23T18:05:05.110265+0000","last_undegraded":"2023-06-23T18:05:05.110265+0000","last_fullsized":"2023-06-23T18:05:05.110265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:45:23.912814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110047+0000","last_change":"2023-06-23T18:05:04.127175+0000","last_active":"2023-06-23T18:05:05.110047+0000","last_peered":"2023-06-23T18:05:05.110047+0000","last_clean":"2023-06-23T18:05:05.110047+0000","last_became_active":"2023-06-23T18:05:04.126875+0000","last_became_peered":"2023-06-23T18:05:04.126875+0000","last_unstale":"2023-06-23T18:05:05.110047+0000","last_undegraded":"2023-06-23T18:05:05.110047+0000","last_fullsized":"2023-06-23T18:05:05.110047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:23:43.253102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110558+0000","last_change":"2023-06-23T18:05:03.727398+0000","last_active":"2023-06-23T18:05:05.110558+0000","last_peered":"2023-06-23T18:05:05.110558+0000","last_clean":"2023-06-23T18:05:05.110558+0000","last_became_active":"2023-06-23T18:05:03.726739+0000","last_became_peered":"2023-06-23T18:05:03.726739+0000","last_unstale":"2023-06-23T18:05:05.110558+0000","last_undegraded":"2023-06-23T18:05:05.110558+0000","last_fullsized":"2023-06-23T18:05:05.110558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:13.976421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438843+0000","last_change":"2023-06-23T18:05:04.122243+0000","last_active":"2023-06-23T18:05:05.438843+0000","last_peered":"2023-06-23T18:05:05.438843+0000","last_clean":"2023-06-23T18:05:05.438843+0000","last_became_active":"2023-06-23T18:05:04.121991+0000","last_became_peered":"2023-06-23T18:05:04.121991+0000","last_unstale":"2023-06-23T18:05:05.438843+0000","last_undegraded":"2023-06-23T18:05:05.438843+0000","last_fullsized":"2023-06-23T18:05:05.438843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:18:35.359050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438930+0000","last_change":"2023-06-23T18:05:03.723358+0000","last_active":"2023-06-23T18:05:05.438930+0000","last_peered":"2023-06-23T18:05:05.438930+0000","last_clean":"2023-06-23T18:05:05.438930+0000","last_became_active":"2023-06-23T18:05:03.723005+0000","last_became_peered":"2023-06-23T18:05:03.723005+0000","last_unstale":"2023-06-23T18:05:05.438930+0000","last_undegraded":"2023-06-23T18:05:05.438930+0000","last_fullsized":"2023-06-23T18:05:05.438930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437747+0000","last_change":"2023-06-23T18:05:04.122156+0000","last_active":"2023-06-23T18:05:05.437747+0000","last_peered":"2023-06-23T18:05:05.437747+0000","last_clean":"2023-06-23T18:05:05.437747+0000","last_became_active":"2023-06-23T18:05:04.121872+0000","last_became_peered":"2023-06-23T18:05:04.121872+0000","last_unstale":"2023-06-23T18:05:05.437747+0000","last_undegraded":"2023-06-23T18:05:05.437747+0000","last_fullsized":"2023-06-23T18:05:05.437747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:01:39.428071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441413+0000","last_change":"2023-06-23T18:05:03.727740+0000","last_active":"2023-06-23T18:05:05.441413+0000","last_peered":"2023-06-23T18:05:05.441413+0000","last_clean":"2023-06-23T18:05:05.441413+0000","last_became_active":"2023-06-23T18:05:03.727377+0000","last_became_peered":"2023-06-23T18:05:03.727377+0000","last_unstale":"2023-06-23T18:05:05.441413+0000","last_undegraded":"2023-06-23T18:05:05.441413+0000","last_fullsized":"2023-06-23T18:05:05.441413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:20.422452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441407+0000","last_change":"2023-06-23T18:05:04.120326+0000","last_active":"2023-06-23T18:05:05.441407+0000","last_peered":"2023-06-23T18:05:05.441407+0000","last_clean":"2023-06-23T18:05:05.441407+0000","last_became_active":"2023-06-23T18:05:04.120027+0000","last_became_peered":"2023-06-23T18:05:04.120027+0000","last_unstale":"2023-06-23T18:05:05.441407+0000","last_undegraded":"2023-06-23T18:05:05.441407+0000","last_fullsized":"2023-06-23T18:05:05.441407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:26.466042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.442036+0000","last_change":"2023-06-23T18:05:03.729752+0000","last_active":"2023-06-23T18:05:05.442036+0000","last_peered":"2023-06-23T18:05:05.442036+0000","last_clean":"2023-06-23T18:05:05.442036+0000","last_became_active":"2023-06-23T18:05:03.729439+0000","last_became_peered":"2023-06-23T18:05:03.729439+0000","last_unstale":"2023-06-23T18:05:05.442036+0000","last_undegraded":"2023-06-23T18:05:05.442036+0000","last_fullsized":"2023-06-23T18:05:05.442036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:02:46.398537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110188+0000","last_change":"2023-06-23T18:05:04.127646+0000","last_active":"2023-06-23T18:05:05.110188+0000","last_peered":"2023-06-23T18:05:05.110188+0000","last_clean":"2023-06-23T18:05:05.110188+0000","last_became_active":"2023-06-23T18:05:04.127167+0000","last_became_peered":"2023-06-23T18:05:04.127167+0000","last_unstale":"2023-06-23T18:05:05.110188+0000","last_undegraded":"2023-06-23T18:05:05.110188+0000","last_fullsized":"2023-06-23T18:05:05.110188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:52:35.317430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.442009+0000","last_change":"2023-06-23T18:05:03.724361+0000","last_active":"2023-06-23T18:05:05.442009+0000","last_peered":"2023-06-23T18:05:05.442009+0000","last_clean":"2023-06-23T18:05:05.442009+0000","last_became_active":"2023-06-23T18:05:03.724064+0000","last_became_peered":"2023-06-23T18:05:03.724064+0000","last_unstale":"2023-06-23T18:05:05.442009+0000","last_undegraded":"2023-06-23T18:05:05.442009+0000","last_fullsized":"2023-06-23T18:05:05.442009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:29:49.123036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439844+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:05:05.439844+0000","last_peered":"2023-06-23T18:05:05.439844+0000","last_clean":"2023-06-23T18:05:05.439844+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:05:05.439844+0000","last_undegraded":"2023-06-23T18:05:05.439844+0000","last_fullsized":"2023-06-23T18:05:05.439844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:27:47.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110335+0000","last_change":"2023-06-23T18:05:04.131076+0000","last_active":"2023-06-23T18:05:05.110335+0000","last_peered":"2023-06-23T18:05:05.110335+0000","last_clean":"2023-06-23T18:05:05.110335+0000","last_became_active":"2023-06-23T18:05:04.126407+0000","last_became_peered":"2023-06-23T18:05:04.126407+0000","last_unstale":"2023-06-23T18:05:05.110335+0000","last_undegraded":"2023-06-23T18:05:05.110335+0000","last_fullsized":"2023-06-23T18:05:05.110335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:15.534113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113266+0000","last_change":"2023-06-23T18:05:03.727717+0000","last_active":"2023-06-23T18:05:05.113266+0000","last_peered":"2023-06-23T18:05:05.113266+0000","last_clean":"2023-06-23T18:05:05.113266+0000","last_became_active":"2023-06-23T18:05:03.727394+0000","last_became_peered":"2023-06-23T18:05:03.727394+0000","last_unstale":"2023-06-23T18:05:05.113266+0000","last_undegraded":"2023-06-23T18:05:05.113266+0000","last_fullsized":"2023-06-23T18:05:05.113266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:58:41.573936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439185+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:05:05.439185+0000","last_peered":"2023-06-23T18:05:05.439185+0000","last_clean":"2023-06-23T18:05:05.439185+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:05:05.439185+0000","last_undegraded":"2023-06-23T18:05:05.439185+0000","last_fullsized":"2023-06-23T18:05:05.439185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:37:47.464339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438096+0000","last_change":"2023-06-23T18:05:04.129424+0000","last_active":"2023-06-23T18:05:05.438096+0000","last_peered":"2023-06-23T18:05:05.438096+0000","last_clean":"2023-06-23T18:05:05.438096+0000","last_became_active":"2023-06-23T18:05:04.128128+0000","last_became_peered":"2023-06-23T18:05:04.128128+0000","last_unstale":"2023-06-23T18:05:05.438096+0000","last_undegraded":"2023-06-23T18:05:05.438096+0000","last_fullsized":"2023-06-23T18:05:05.438096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:42.099943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439282+0000","last_change":"2023-06-23T18:05:03.726337+0000","last_active":"2023-06-23T18:05:05.439282+0000","last_peered":"2023-06-23T18:05:05.439282+0000","last_clean":"2023-06-23T18:05:05.439282+0000","last_became_active":"2023-06-23T18:05:03.726012+0000","last_became_peered":"2023-06-23T18:05:03.726012+0000","last_unstale":"2023-06-23T18:05:05.439282+0000","last_undegraded":"2023-06-23T18:05:05.439282+0000","last_fullsized":"2023-06-23T18:05:05.439282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:41:14.934399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441985+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:05:05.441985+0000","last_peered":"2023-06-23T18:05:05.441985+0000","last_clean":"2023-06-23T18:05:05.441985+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:05:05.441985+0000","last_undegraded":"2023-06-23T18:05:05.441985+0000","last_fullsized":"2023-06-23T18:05:05.441985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:09.384203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.112886+0000","last_change":"2023-06-23T18:05:04.125214+0000","last_active":"2023-06-23T18:05:05.112886+0000","last_peered":"2023-06-23T18:05:05.112886+0000","last_clean":"2023-06-23T18:05:05.112886+0000","last_became_active":"2023-06-23T18:05:04.124802+0000","last_became_peered":"2023-06-23T18:05:04.124802+0000","last_unstale":"2023-06-23T18:05:05.112886+0000","last_undegraded":"2023-06-23T18:05:05.112886+0000","last_fullsized":"2023-06-23T18:05:05.112886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:25.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440728+0000","last_change":"2023-06-23T18:05:03.727367+0000","last_active":"2023-06-23T18:05:05.440728+0000","last_peered":"2023-06-23T18:05:05.440728+0000","last_clean":"2023-06-23T18:05:05.440728+0000","last_became_active":"2023-06-23T18:05:03.726518+0000","last_became_peered":"2023-06-23T18:05:03.726518+0000","last_unstale":"2023-06-23T18:05:05.440728+0000","last_undegraded":"2023-06-23T18:05:05.440728+0000","last_fullsized":"2023-06-23T18:05:05.440728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:41.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438233+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:05:05.438233+0000","last_peered":"2023-06-23T18:05:05.438233+0000","last_clean":"2023-06-23T18:05:05.438233+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:05:05.438233+0000","last_undegraded":"2023-06-23T18:05:05.438233+0000","last_fullsized":"2023-06-23T18:05:05.438233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:29:07.778826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439662+0000","last_change":"2023-06-23T18:05:04.129560+0000","last_active":"2023-06-23T18:05:05.439662+0000","last_peered":"2023-06-23T18:05:05.439662+0000","last_clean":"2023-06-23T18:05:05.439662+0000","last_became_active":"2023-06-23T18:05:04.129254+0000","last_became_peered":"2023-06-23T18:05:04.129254+0000","last_unstale":"2023-06-23T18:05:05.439662+0000","last_undegraded":"2023-06-23T18:05:05.439662+0000","last_fullsized":"2023-06-23T18:05:05.439662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:10:35.937401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110513+0000","last_change":"2023-06-23T18:05:03.732330+0000","last_active":"2023-06-23T18:05:05.110513+0000","last_peered":"2023-06-23T18:05:05.110513+0000","last_clean":"2023-06-23T18:05:05.110513+0000","last_became_active":"2023-06-23T18:05:03.732034+0000","last_became_peered":"2023-06-23T18:05:03.732034+0000","last_unstale":"2023-06-23T18:05:05.110513+0000","last_undegraded":"2023-06-23T18:05:05.110513+0000","last_fullsized":"2023-06-23T18:05:05.110513+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:53.415660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439127+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:05:05.439127+0000","last_peered":"2023-06-23T18:05:05.439127+0000","last_clean":"2023-06-23T18:05:05.439127+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:05:05.439127+0000","last_undegraded":"2023-06-23T18:05:05.439127+0000","last_fullsized":"2023-06-23T18:05:05.439127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:50:46.322252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440130+0000","last_change":"2023-06-23T18:05:04.118390+0000","last_active":"2023-06-23T18:05:05.440130+0000","last_peered":"2023-06-23T18:05:05.440130+0000","last_clean":"2023-06-23T18:05:05.440130+0000","last_became_active":"2023-06-23T18:05:04.118051+0000","last_became_peered":"2023-06-23T18:05:04.118051+0000","last_unstale":"2023-06-23T18:05:05.440130+0000","last_undegraded":"2023-06-23T18:05:05.440130+0000","last_fullsized":"2023-06-23T18:05:05.440130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:42.099943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441882+0000","last_change":"2023-06-23T18:05:03.724378+0000","last_active":"2023-06-23T18:05:05.441882+0000","last_peered":"2023-06-23T18:05:05.441882+0000","last_clean":"2023-06-23T18:05:05.441882+0000","last_became_active":"2023-06-23T18:05:03.724073+0000","last_became_peered":"2023-06-23T18:05:03.724073+0000","last_unstale":"2023-06-23T18:05:05.441882+0000","last_undegraded":"2023-06-23T18:05:05.441882+0000","last_fullsized":"2023-06-23T18:05:05.441882+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:11:20.986246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437278+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:05:05.437278+0000","last_peered":"2023-06-23T18:05:05.437278+0000","last_clean":"2023-06-23T18:05:05.437278+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:05:05.437278+0000","last_undegraded":"2023-06-23T18:05:05.437278+0000","last_fullsized":"2023-06-23T18:05:05.437278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:12:57.127001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441600+0000","last_change":"2023-06-23T18:05:04.119695+0000","last_active":"2023-06-23T18:05:05.441600+0000","last_peered":"2023-06-23T18:05:05.441600+0000","last_clean":"2023-06-23T18:05:05.441600+0000","last_became_active":"2023-06-23T18:05:04.119404+0000","last_became_peered":"2023-06-23T18:05:04.119404+0000","last_unstale":"2023-06-23T18:05:05.441600+0000","last_undegraded":"2023-06-23T18:05:05.441600+0000","last_fullsized":"2023-06-23T18:05:05.441600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438019+0000","last_change":"2023-06-23T18:05:03.726382+0000","last_active":"2023-06-23T18:05:05.438019+0000","last_peered":"2023-06-23T18:05:05.438019+0000","last_clean":"2023-06-23T18:05:05.438019+0000","last_became_active":"2023-06-23T18:05:03.725770+0000","last_became_peered":"2023-06-23T18:05:03.725770+0000","last_unstale":"2023-06-23T18:05:05.438019+0000","last_undegraded":"2023-06-23T18:05:05.438019+0000","last_fullsized":"2023-06-23T18:05:05.438019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:56:28.102729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440955+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:05:05.440955+0000","last_peered":"2023-06-23T18:05:05.440955+0000","last_clean":"2023-06-23T18:05:05.440955+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:05:05.440955+0000","last_undegraded":"2023-06-23T18:05:05.440955+0000","last_fullsized":"2023-06-23T18:05:05.440955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:39.355724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441109+0000","last_change":"2023-06-23T18:05:04.129350+0000","last_active":"2023-06-23T18:05:05.441109+0000","last_peered":"2023-06-23T18:05:05.441109+0000","last_clean":"2023-06-23T18:05:05.441109+0000","last_became_active":"2023-06-23T18:05:04.127874+0000","last_became_peered":"2023-06-23T18:05:04.127874+0000","last_unstale":"2023-06-23T18:05:05.441109+0000","last_undegraded":"2023-06-23T18:05:05.441109+0000","last_fullsized":"2023-06-23T18:05:05.441109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440697+0000","last_change":"2023-06-23T18:05:03.726812+0000","last_active":"2023-06-23T18:05:05.440697+0000","last_peered":"2023-06-23T18:05:05.440697+0000","last_clean":"2023-06-23T18:05:05.440697+0000","last_became_active":"2023-06-23T18:05:03.726544+0000","last_became_peered":"2023-06-23T18:05:03.726544+0000","last_unstale":"2023-06-23T18:05:05.440697+0000","last_undegraded":"2023-06-23T18:05:05.440697+0000","last_fullsized":"2023-06-23T18:05:05.440697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:08.542108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439455+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:05:05.439455+0000","last_peered":"2023-06-23T18:05:05.439455+0000","last_clean":"2023-06-23T18:05:05.439455+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:05:05.439455+0000","last_undegraded":"2023-06-23T18:05:05.439455+0000","last_fullsized":"2023-06-23T18:05:05.439455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:11.402900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441122+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:05:05.441122+0000","last_peered":"2023-06-23T18:05:05.441122+0000","last_clean":"2023-06-23T18:05:05.441122+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:05:05.441122+0000","last_undegraded":"2023-06-23T18:05:05.441122+0000","last_fullsized":"2023-06-23T18:05:05.441122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:50:46.322252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":1048576,"data_stored":794055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.0069999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.175},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0389999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":458752,"data_stored":204269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0129999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.96199999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.093}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.98299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":454656,"data_stored":203667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":1.038}]},{"osd":2,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.871}]},{"osd":3,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.76200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.099},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.1459999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0169999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":462848,"data_stored":205161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.518},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.556},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.6259999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.3180000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.145},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.673}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":462848,"data_stored":204235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.629}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.246}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1559999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31504,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340336,"statfs":{"total":96636764160,"available":96604504064,"internally_reserved":0,"allocated":466944,"data_stored":205183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78400000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.94899999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.079}]},{"osd":4,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.89400000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.264},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.1220000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.024},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.218},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31492,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340348,"statfs":{"total":96636764160,"available":96604516352,"internally_reserved":0,"allocated":454656,"data_stored":203667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.502}]},{"osd":2,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1048576,"data_stored":793999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.216}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.333},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.117},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.022}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.159},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.091}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.29},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.258}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:05:08.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:05:08.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.539+0000 7fb4f66e1700 1 Processor -- start 2023-06-23T18:05:08.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.539+0000 7fb4f66e1700 1 -- start start 2023-06-23T18:05:08.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 0x7fb4f00c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4f00ccd70 0x7fb4f0163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f00cc360 con 0x7fb4f0163f90 2023-06-23T18:05:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00cdb60 con 0x7fb4f00ccd70 2023-06-23T18:05:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01643b0 con 0x7fb4f015ff90 2023-06-23T18:05:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f56df700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 0x7fb4f00c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4f00ccd70 0x7fb4f0163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4f00ccd70 0x7fb4f0163a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46812/0 (socket says 172.21.15.22:46812) 2023-06-23T18:05:08.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f56df700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 0x7fb4f00c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:50306/0 (socket says 172.21.15.22:50306) 2023-06-23T18:05:08.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 -- 172.21.15.22:0/2491852606 learned_addr learned my addr 172.21.15.22:0/2491852606 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f5ee0700 1 --1- 172.21.15.22:0/2491852606 >> v1:172.21.15.169:6789/0 conn(0x7fb4f0163f90 0x7fb4f006a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.22:40840/0 (socket says 172.21.15.22:40840) 2023-06-23T18:05:08.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 -- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 msgr2=0x7fb4f00c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:08.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 --2- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 0x7fb4f00c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 -- 172.21.15.22:0/2491852606 >> v1:172.21.15.169:6789/0 conn(0x7fb4f0163f90 legacy=0x7fb4f006a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 -- 172.21.15.22:0/2491852606 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0160360 con 0x7fb4f00ccd70 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f4ede700 1 --2- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f00ccd70 0x7fb4f0163a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009910 tx=0x7fb4ec009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e424006b5ea29c14 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4e3fff700 1 -- 172.21.15.22:0/2491852606 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011830 con 0x7fb4f00ccd70 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4e3fff700 1 -- 172.21.15.22:0/2491852606 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec011990 con 0x7fb4f00ccd70 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4e3fff700 1 -- 172.21.15.22:0/2491852606 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011d40 con 0x7fb4f00ccd70 2023-06-23T18:05:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f00ccd70 msgr2=0x7fb4f0163a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f00ccd70 0x7fb4f0163a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009910 tx=0x7fb4ec009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- 172.21.15.22:0/2491852606 shutdown_connections 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f00ccd70 0x7fb4f0163a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/2491852606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f015ff90 0x7fb4f00c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- 172.21.15.22:0/2491852606 >> 172.21.15.22:0/2491852606 conn(0x7fb4f00cb2a0 msgr2=0x7fb4f00cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- 172.21.15.22:0/2491852606 shutdown_connections 2023-06-23T18:05:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.543+0000 7fb4f66e1700 1 -- 172.21.15.22:0/2491852606 wait complete. 2023-06-23T18:05:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 Processor -- start 2023-06-23T18:05:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 -- start start 2023-06-23T18:05:08.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 0x7fb4f0160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f015ff90 0x7fb4f0153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01542f0 con 0x7fb4f0163f90 2023-06-23T18:05:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0161870 con 0x7fb4f015ff90 2023-06-23T18:05:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f66e1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01631b0 con 0x7fb4f00ccd70 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f5ee0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 0x7fb4f0160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f5ee0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44570/0 (socket says 172.21.15.22:44570) 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f5ee0700 1 -- 172.21.15.22:0/4048185680 learned_addr learned my addr 172.21.15.22:0/4048185680 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f4ede700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f015ff90 0x7fb4f0153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 -- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f015ff90 msgr2=0x7fb4f0153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f015ff90 0x7fb4f0153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 -- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 msgr2=0x7fb4f0162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f56df700 1 -- 172.21.15.22:0/4048185680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec009570 con 0x7fb4f00ccd70 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.547+0000 7fb4f5ee0700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4f56df700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 0x7fb4f0160550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800ebf0 tx=0x7fb4e800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800ca80 con 0x7fb4f00ccd70 2023-06-23T18:05:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e800f040 con 0x7fb4f00ccd70 2023-06-23T18:05:08.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0160c60 con 0x7fb4f00ccd70 2023-06-23T18:05:08.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800ce20 con 0x7fb4f00ccd70 2023-06-23T18:05:08.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f0161220 con 0x7fb4f00ccd70 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f004e4c0 con 0x7fb4f00ccd70 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.551+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb4e8014480 con 0x7fb4f00ccd70 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.555+0000 7fb4e1ffb700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 0x7fb4d40687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.555+0000 7fb4f4ede700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 0x7fb4d40687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.555+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb4e8084930 con 0x7fb4f00ccd70 2023-06-23T18:05:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.555+0000 7fb4f4ede700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 0x7fb4d40687a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009c60 tx=0x7fb4ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.563+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb4e8051b50 con 0x7fb4f00ccd70 2023-06-23T18:05:08.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.727+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 --> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f00cd2e0 con 0x7fb4d40662f0 2023-06-23T18:05:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.743+0000 7fb4e1ffb700 1 -- 172.21.15.22:0/4048185680 <== mgr.4108 v2:172.21.15.22:6832/3032843375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309868 (secure 0 0 0) 0x7fb4ec011b00 con 0x7fb4d40662f0 2023-06-23T18:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 msgr2=0x7fb4d40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 0x7fb4d40687a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009c60 tx=0x7fb4ec008030 comp rx=0 tx=0).stop 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 msgr2=0x7fb4f0160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 0x7fb4f0160550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800ebf0 tx=0x7fb4e800ef00 comp rx=0 tx=0).stop 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 shutdown_connections 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4f0163f90 0x7fb4f0162c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb4d40662f0 0x7fb4d40687a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f015ff90 0x7fb4f0153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 --2- 172.21.15.22:0/4048185680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00ccd70 0x7fb4f0160550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 >> 172.21.15.22:0/4048185680 conn(0x7fb4f00cb2a0 msgr2=0x7fb4f00b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 shutdown_connections 2023-06-23T18:05:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.751+0000 7fb4f66e1700 1 -- 172.21.15.22:0/4048185680 wait complete. 2023-06-23T18:05:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-06-23T18:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-06-23T18:05:07.638722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236744,"kb_used_data":4744,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754737976,"statfs":{"total":773094113280,"available":772851687424,"internally_reserved":0,"allocated":4857856,"data_stored":2814236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942624"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441621+0000","last_change":"2023-06-23T18:05:04.120120+0000","last_active":"2023-06-23T18:05:05.441621+0000","last_peered":"2023-06-23T18:05:05.441621+0000","last_clean":"2023-06-23T18:05:05.441621+0000","last_became_active":"2023-06-23T18:05:04.119656+0000","last_became_peered":"2023-06-23T18:05:04.119656+0000","last_unstale":"2023-06-23T18:05:05.441621+0000","last_undegraded":"2023-06-23T18:05:05.441621+0000","last_fullsized":"2023-06-23T18:05:05.441621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:56.303413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440417+0000","last_change":"2023-06-23T18:05:04.136281+0000","last_active":"2023-06-23T18:05:05.440417+0000","last_peered":"2023-06-23T18:05:05.440417+0000","last_clean":"2023-06-23T18:05:05.440417+0000","last_became_active":"2023-06-23T18:05:04.136011+0000","last_became_peered":"2023-06-23T18:05:04.136011+0000","last_unstale":"2023-06-23T18:05:05.440417+0000","last_undegraded":"2023-06-23T18:05:05.440417+0000","last_fullsized":"2023-06-23T18:05:05.440417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:21.426316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113668+0000","last_change":"2023-06-23T18:05:04.119079+0000","last_active":"2023-06-23T18:05:05.113668+0000","last_peered":"2023-06-23T18:05:05.113668+0000","last_clean":"2023-06-23T18:05:05.113668+0000","last_became_active":"2023-06-23T18:05:04.118752+0000","last_became_peered":"2023-06-23T18:05:04.118752+0000","last_unstale":"2023-06-23T18:05:05.113668+0000","last_undegraded":"2023-06-23T18:05:05.113668+0000","last_fullsized":"2023-06-23T18:05:05.113668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:51:11.597188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440044+0000","last_change":"2023-06-23T18:05:04.135013+0000","last_active":"2023-06-23T18:05:05.440044+0000","last_peered":"2023-06-23T18:05:05.440044+0000","last_clean":"2023-06-23T18:05:05.440044+0000","last_became_active":"2023-06-23T18:05:04.134744+0000","last_became_peered":"2023-06-23T18:05:04.134744+0000","last_unstale":"2023-06-23T18:05:05.440044+0000","last_undegraded":"2023-06-23T18:05:05.440044+0000","last_fullsized":"2023-06-23T18:05:05.440044+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:54.419525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439278+0000","last_change":"2023-06-23T18:05:04.136379+0000","last_active":"2023-06-23T18:05:05.439278+0000","last_peered":"2023-06-23T18:05:05.439278+0000","last_clean":"2023-06-23T18:05:05.439278+0000","last_became_active":"2023-06-23T18:05:04.136050+0000","last_became_peered":"2023-06-23T18:05:04.136050+0000","last_unstale":"2023-06-23T18:05:05.439278+0000","last_undegraded":"2023-06-23T18:05:05.439278+0000","last_fullsized":"2023-06-23T18:05:05.439278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:26.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110207+0000","last_change":"2023-06-23T18:05:04.124092+0000","last_active":"2023-06-23T18:05:05.110207+0000","last_peered":"2023-06-23T18:05:05.110207+0000","last_clean":"2023-06-23T18:05:05.110207+0000","last_became_active":"2023-06-23T18:05:04.123481+0000","last_became_peered":"2023-06-23T18:05:04.123481+0000","last_unstale":"2023-06-23T18:05:05.110207+0000","last_undegraded":"2023-06-23T18:05:05.110207+0000","last_fullsized":"2023-06-23T18:05:05.110207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:17:03.193671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441539+0000","last_change":"2023-06-23T18:05:04.124289+0000","last_active":"2023-06-23T18:05:05.441539+0000","last_peered":"2023-06-23T18:05:05.441539+0000","last_clean":"2023-06-23T18:05:05.441539+0000","last_became_active":"2023-06-23T18:05:04.124058+0000","last_became_peered":"2023-06-23T18:05:04.124058+0000","last_unstale":"2023-06-23T18:05:05.441539+0000","last_undegraded":"2023-06-23T18:05:05.441539+0000","last_fullsized":"2023-06-23T18:05:05.441539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440130+0000","last_change":"2023-06-23T18:05:04.135114+0000","last_active":"2023-06-23T18:05:05.440130+0000","last_peered":"2023-06-23T18:05:05.440130+0000","last_clean":"2023-06-23T18:05:05.440130+0000","last_became_active":"2023-06-23T18:05:04.134739+0000","last_became_peered":"2023-06-23T18:05:04.134739+0000","last_unstale":"2023-06-23T18:05:05.440130+0000","last_undegraded":"2023-06-23T18:05:05.440130+0000","last_fullsized":"2023-06-23T18:05:05.440130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:14.980285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438666+0000","last_change":"2023-06-23T18:05:04.134531+0000","last_active":"2023-06-23T18:05:05.438666+0000","last_peered":"2023-06-23T18:05:05.438666+0000","last_clean":"2023-06-23T18:05:05.438666+0000","last_became_active":"2023-06-23T18:05:04.134220+0000","last_became_peered":"2023-06-23T18:05:04.134220+0000","last_unstale":"2023-06-23T18:05:05.438666+0000","last_undegraded":"2023-06-23T18:05:05.438666+0000","last_fullsized":"2023-06-23T18:05:05.438666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:23:43.253102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441086+0000","last_change":"2023-06-23T18:05:04.130561+0000","last_active":"2023-06-23T18:05:05.441086+0000","last_peered":"2023-06-23T18:05:05.441086+0000","last_clean":"2023-06-23T18:05:05.441086+0000","last_became_active":"2023-06-23T18:05:04.130303+0000","last_became_peered":"2023-06-23T18:05:04.130303+0000","last_unstale":"2023-06-23T18:05:05.441086+0000","last_undegraded":"2023-06-23T18:05:05.441086+0000","last_fullsized":"2023-06-23T18:05:05.441086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439913+0000","last_change":"2023-06-23T18:05:04.124947+0000","last_active":"2023-06-23T18:05:05.439913+0000","last_peered":"2023-06-23T18:05:05.439913+0000","last_clean":"2023-06-23T18:05:05.439913+0000","last_became_active":"2023-06-23T18:05:04.124698+0000","last_became_peered":"2023-06-23T18:05:04.124698+0000","last_unstale":"2023-06-23T18:05:05.439913+0000","last_undegraded":"2023-06-23T18:05:05.439913+0000","last_fullsized":"2023-06-23T18:05:05.439913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:15.534113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439984+0000","last_change":"2023-06-23T18:05:04.135170+0000","last_active":"2023-06-23T18:05:05.439984+0000","last_peered":"2023-06-23T18:05:05.439984+0000","last_clean":"2023-06-23T18:05:05.439984+0000","last_became_active":"2023-06-23T18:05:04.134897+0000","last_became_peered":"2023-06-23T18:05:04.134897+0000","last_unstale":"2023-06-23T18:05:05.439984+0000","last_undegraded":"2023-06-23T18:05:05.439984+0000","last_fullsized":"2023-06-23T18:05:05.439984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:50:12.781962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438109+0000","last_change":"2023-06-23T18:05:04.124468+0000","last_active":"2023-06-23T18:05:05.438109+0000","last_peered":"2023-06-23T18:05:05.438109+0000","last_clean":"2023-06-23T18:05:05.438109+0000","last_became_active":"2023-06-23T18:05:04.124222+0000","last_became_peered":"2023-06-23T18:05:04.124222+0000","last_unstale":"2023-06-23T18:05:05.438109+0000","last_undegraded":"2023-06-23T18:05:05.438109+0000","last_fullsized":"2023-06-23T18:05:05.438109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:09:59.421209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439671+0000","last_change":"2023-06-23T18:05:04.119644+0000","last_active":"2023-06-23T18:05:05.439671+0000","last_peered":"2023-06-23T18:05:05.439671+0000","last_clean":"2023-06-23T18:05:05.439671+0000","last_became_active":"2023-06-23T18:05:04.119384+0000","last_became_peered":"2023-06-23T18:05:04.119384+0000","last_unstale":"2023-06-23T18:05:05.439671+0000","last_undegraded":"2023-06-23T18:05:05.439671+0000","last_fullsized":"2023-06-23T18:05:05.439671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:27:28.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.109772+0000","last_change":"2023-06-23T18:05:04.123945+0000","last_active":"2023-06-23T18:05:05.109772+0000","last_peered":"2023-06-23T18:05:05.109772+0000","last_clean":"2023-06-23T18:05:05.109772+0000","last_became_active":"2023-06-23T18:05:04.123620+0000","last_became_peered":"2023-06-23T18:05:04.123620+0000","last_unstale":"2023-06-23T18:05:05.109772+0000","last_undegraded":"2023-06-23T18:05:05.109772+0000","last_fullsized":"2023-06-23T18:05:05.109772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:31.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439710+0000","last_change":"2023-06-23T18:05:04.136041+0000","last_active":"2023-06-23T18:05:05.439710+0000","last_peered":"2023-06-23T18:05:05.439710+0000","last_clean":"2023-06-23T18:05:05.439710+0000","last_became_active":"2023-06-23T18:05:04.135746+0000","last_became_peered":"2023-06-23T18:05:04.135746+0000","last_unstale":"2023-06-23T18:05:05.439710+0000","last_undegraded":"2023-06-23T18:05:05.439710+0000","last_fullsized":"2023-06-23T18:05:05.439710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:29.436604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113343+0000","last_change":"2023-06-23T18:05:04.123364+0000","last_active":"2023-06-23T18:05:05.113343+0000","last_peered":"2023-06-23T18:05:05.113343+0000","last_clean":"2023-06-23T18:05:05.113343+0000","last_became_active":"2023-06-23T18:05:04.123005+0000","last_became_peered":"2023-06-23T18:05:04.123005+0000","last_unstale":"2023-06-23T18:05:05.113343+0000","last_undegraded":"2023-06-23T18:05:05.113343+0000","last_fullsized":"2023-06-23T18:05:05.113343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:09:27.091323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113136+0000","last_change":"2023-06-23T18:05:04.136249+0000","last_active":"2023-06-23T18:05:05.113136+0000","last_peered":"2023-06-23T18:05:05.113136+0000","last_clean":"2023-06-23T18:05:05.113136+0000","last_became_active":"2023-06-23T18:05:04.135943+0000","last_became_peered":"2023-06-23T18:05:04.135943+0000","last_unstale":"2023-06-23T18:05:05.113136+0000","last_undegraded":"2023-06-23T18:05:05.113136+0000","last_fullsized":"2023-06-23T18:05:05.113136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:02.966525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439140+0000","last_change":"2023-06-23T18:05:04.136717+0000","last_active":"2023-06-23T18:05:05.439140+0000","last_peered":"2023-06-23T18:05:05.439140+0000","last_clean":"2023-06-23T18:05:05.439140+0000","last_became_active":"2023-06-23T18:05:04.136439+0000","last_became_peered":"2023-06-23T18:05:04.136439+0000","last_unstale":"2023-06-23T18:05:05.439140+0000","last_undegraded":"2023-06-23T18:05:05.439140+0000","last_fullsized":"2023-06-23T18:05:05.439140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:05:22.128448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439200+0000","last_change":"2023-06-23T18:05:04.123307+0000","last_active":"2023-06-23T18:05:05.439200+0000","last_peered":"2023-06-23T18:05:05.439200+0000","last_clean":"2023-06-23T18:05:05.439200+0000","last_became_active":"2023-06-23T18:05:04.123100+0000","last_became_peered":"2023-06-23T18:05:04.123100+0000","last_unstale":"2023-06-23T18:05:05.439200+0000","last_undegraded":"2023-06-23T18:05:05.439200+0000","last_fullsized":"2023-06-23T18:05:05.439200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:46:43.742354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.109990+0000","last_change":"2023-06-23T18:05:04.131916+0000","last_active":"2023-06-23T18:05:05.109990+0000","last_peered":"2023-06-23T18:05:05.109990+0000","last_clean":"2023-06-23T18:05:05.109990+0000","last_became_active":"2023-06-23T18:05:04.131334+0000","last_became_peered":"2023-06-23T18:05:04.131334+0000","last_unstale":"2023-06-23T18:05:05.109990+0000","last_undegraded":"2023-06-23T18:05:05.109990+0000","last_fullsized":"2023-06-23T18:05:05.109990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:26:00.815297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439652+0000","last_change":"2023-06-23T18:05:04.130365+0000","last_active":"2023-06-23T18:05:05.439652+0000","last_peered":"2023-06-23T18:05:05.439652+0000","last_clean":"2023-06-23T18:05:05.439652+0000","last_became_active":"2023-06-23T18:05:04.130012+0000","last_became_peered":"2023-06-23T18:05:04.130012+0000","last_unstale":"2023-06-23T18:05:05.439652+0000","last_undegraded":"2023-06-23T18:05:05.439652+0000","last_fullsized":"2023-06-23T18:05:05.439652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:45:24.916678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438862+0000","last_change":"2023-06-23T18:05:04.124423+0000","last_active":"2023-06-23T18:05:05.438862+0000","last_peered":"2023-06-23T18:05:05.438862+0000","last_clean":"2023-06-23T18:05:05.438862+0000","last_became_active":"2023-06-23T18:05:04.124106+0000","last_became_peered":"2023-06-23T18:05:04.124106+0000","last_unstale":"2023-06-23T18:05:05.438862+0000","last_undegraded":"2023-06-23T18:05:05.438862+0000","last_fullsized":"2023-06-23T18:05:05.438862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:28.379296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438694+0000","last_change":"2023-06-23T18:05:04.123494+0000","last_active":"2023-06-23T18:05:05.438694+0000","last_peered":"2023-06-23T18:05:05.438694+0000","last_clean":"2023-06-23T18:05:05.438694+0000","last_became_active":"2023-06-23T18:05:04.123196+0000","last_became_peered":"2023-06-23T18:05:04.123196+0000","last_unstale":"2023-06-23T18:05:05.438694+0000","last_undegraded":"2023-06-23T18:05:05.438694+0000","last_fullsized":"2023-06-23T18:05:05.438694+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:29:03.099380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439297+0000","last_change":"2023-06-23T18:05:04.132615+0000","last_active":"2023-06-23T18:05:05.439297+0000","last_peered":"2023-06-23T18:05:05.439297+0000","last_clean":"2023-06-23T18:05:05.439297+0000","last_became_active":"2023-06-23T18:05:04.130611+0000","last_became_peered":"2023-06-23T18:05:04.130611+0000","last_unstale":"2023-06-23T18:05:05.439297+0000","last_undegraded":"2023-06-23T18:05:05.439297+0000","last_fullsized":"2023-06-23T18:05:05.439297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:55:39.715711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438094+0000","last_change":"2023-06-23T18:05:04.123816+0000","last_active":"2023-06-23T18:05:05.438094+0000","last_peered":"2023-06-23T18:05:05.438094+0000","last_clean":"2023-06-23T18:05:05.438094+0000","last_became_active":"2023-06-23T18:05:04.123570+0000","last_became_peered":"2023-06-23T18:05:04.123570+0000","last_unstale":"2023-06-23T18:05:05.438094+0000","last_undegraded":"2023-06-23T18:05:05.438094+0000","last_fullsized":"2023-06-23T18:05:05.438094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:47.889106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113850+0000","last_change":"2023-06-23T18:05:04.123820+0000","last_active":"2023-06-23T18:05:05.113850+0000","last_peered":"2023-06-23T18:05:05.113850+0000","last_clean":"2023-06-23T18:05:05.113850+0000","last_became_active":"2023-06-23T18:05:04.123508+0000","last_became_peered":"2023-06-23T18:05:04.123508+0000","last_unstale":"2023-06-23T18:05:05.113850+0000","last_undegraded":"2023-06-23T18:05:05.113850+0000","last_fullsized":"2023-06-23T18:05:05.113850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:41:51.703626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440184+0000","last_change":"2023-06-23T18:05:04.135566+0000","last_active":"2023-06-23T18:05:05.440184+0000","last_peered":"2023-06-23T18:05:05.440184+0000","last_clean":"2023-06-23T18:05:05.440184+0000","last_became_active":"2023-06-23T18:05:04.135286+0000","last_became_peered":"2023-06-23T18:05:04.135286+0000","last_unstale":"2023-06-23T18:05:05.440184+0000","last_undegraded":"2023-06-23T18:05:05.440184+0000","last_fullsized":"2023-06-23T18:05:05.440184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439356+0000","last_change":"2023-06-23T18:05:04.124948+0000","last_active":"2023-06-23T18:05:05.439356+0000","last_peered":"2023-06-23T18:05:05.439356+0000","last_clean":"2023-06-23T18:05:05.439356+0000","last_became_active":"2023-06-23T18:05:04.124675+0000","last_became_peered":"2023-06-23T18:05:04.124675+0000","last_unstale":"2023-06-23T18:05:05.439356+0000","last_undegraded":"2023-06-23T18:05:05.439356+0000","last_fullsized":"2023-06-23T18:05:05.439356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:38:31.217898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.112867+0000","last_change":"2023-06-23T18:05:04.120917+0000","last_active":"2023-06-23T18:05:05.112867+0000","last_peered":"2023-06-23T18:05:05.112867+0000","last_clean":"2023-06-23T18:05:05.112867+0000","last_became_active":"2023-06-23T18:05:04.120639+0000","last_became_peered":"2023-06-23T18:05:04.120639+0000","last_unstale":"2023-06-23T18:05:05.112867+0000","last_undegraded":"2023-06-23T18:05:05.112867+0000","last_fullsized":"2023-06-23T18:05:05.112867+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:53:32.131393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110032+0000","last_change":"2023-06-23T18:05:04.123746+0000","last_active":"2023-06-23T18:05:05.110032+0000","last_peered":"2023-06-23T18:05:05.110032+0000","last_clean":"2023-06-23T18:05:05.110032+0000","last_became_active":"2023-06-23T18:05:04.123436+0000","last_became_peered":"2023-06-23T18:05:04.123436+0000","last_unstale":"2023-06-23T18:05:05.110032+0000","last_undegraded":"2023-06-23T18:05:05.110032+0000","last_fullsized":"2023-06-23T18:05:05.110032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:41:15.938263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437424+0000","last_change":"2023-06-23T18:05:04.114783+0000","last_active":"2023-06-23T18:05:05.437424+0000","last_peered":"2023-06-23T18:05:05.437424+0000","last_clean":"2023-06-23T18:05:05.437424+0000","last_became_active":"2023-06-23T18:05:04.114523+0000","last_became_peered":"2023-06-23T18:05:04.114523+0000","last_unstale":"2023-06-23T18:05:05.437424+0000","last_undegraded":"2023-06-23T18:05:05.437424+0000","last_fullsized":"2023-06-23T18:05:05.437424+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:02:36.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438848+0000","last_change":"2023-06-23T18:05:04.123924+0000","last_active":"2023-06-23T18:05:05.438848+0000","last_peered":"2023-06-23T18:05:05.438848+0000","last_clean":"2023-06-23T18:05:05.438848+0000","last_became_active":"2023-06-23T18:05:04.123682+0000","last_became_peered":"2023-06-23T18:05:04.123682+0000","last_unstale":"2023-06-23T18:05:05.438848+0000","last_undegraded":"2023-06-23T18:05:05.438848+0000","last_fullsized":"2023-06-23T18:05:05.438848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:25:31.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110526+0000","last_change":"2023-06-23T18:05:03.728918+0000","last_active":"2023-06-23T18:05:05.110526+0000","last_peered":"2023-06-23T18:05:05.110526+0000","last_clean":"2023-06-23T18:05:05.110526+0000","last_became_active":"2023-06-23T18:05:03.728637+0000","last_became_peered":"2023-06-23T18:05:03.728637+0000","last_unstale":"2023-06-23T18:05:05.110526+0000","last_undegraded":"2023-06-23T18:05:05.110526+0000","last_fullsized":"2023-06-23T18:05:05.110526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:41.096078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439479+0000","last_change":"2023-06-23T18:05:04.131916+0000","last_active":"2023-06-23T18:05:05.439479+0000","last_peered":"2023-06-23T18:05:05.439479+0000","last_clean":"2023-06-23T18:05:05.439479+0000","last_became_active":"2023-06-23T18:05:04.131556+0000","last_became_peered":"2023-06-23T18:05:04.131556+0000","last_unstale":"2023-06-23T18:05:05.439479+0000","last_undegraded":"2023-06-23T18:05:05.439479+0000","last_fullsized":"2023-06-23T18:05:05.439479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:51.742598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113448+0000","last_change":"2023-06-23T18:05:03.723617+0000","last_active":"2023-06-23T18:05:05.113448+0000","last_peered":"2023-06-23T18:05:05.113448+0000","last_clean":"2023-06-23T18:05:05.113448+0000","last_became_active":"2023-06-23T18:05:03.723356+0000","last_became_peered":"2023-06-23T18:05:03.723356+0000","last_unstale":"2023-06-23T18:05:05.113448+0000","last_undegraded":"2023-06-23T18:05:05.113448+0000","last_fullsized":"2023-06-23T18:05:05.113448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:15:36.304210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440219+0000","last_change":"2023-06-23T18:05:04.128971+0000","last_active":"2023-06-23T18:05:05.440219+0000","last_peered":"2023-06-23T18:05:05.440219+0000","last_clean":"2023-06-23T18:05:05.440219+0000","last_became_active":"2023-06-23T18:05:04.128263+0000","last_became_peered":"2023-06-23T18:05:04.128263+0000","last_unstale":"2023-06-23T18:05:05.440219+0000","last_undegraded":"2023-06-23T18:05:05.440219+0000","last_fullsized":"2023-06-23T18:05:05.440219+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:56.303413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439160+0000","last_change":"2023-06-23T18:05:03.724885+0000","last_active":"2023-06-23T18:05:05.439160+0000","last_peered":"2023-06-23T18:05:05.439160+0000","last_clean":"2023-06-23T18:05:05.439160+0000","last_became_active":"2023-06-23T18:05:03.724514+0000","last_became_peered":"2023-06-23T18:05:03.724514+0000","last_unstale":"2023-06-23T18:05:05.439160+0000","last_undegraded":"2023-06-23T18:05:05.439160+0000","last_fullsized":"2023-06-23T18:05:05.439160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:25:59.811433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438814+0000","last_change":"2023-06-23T18:05:04.130399+0000","last_active":"2023-06-23T18:05:05.438814+0000","last_peered":"2023-06-23T18:05:05.438814+0000","last_clean":"2023-06-23T18:05:05.438814+0000","last_became_active":"2023-06-23T18:05:04.130108+0000","last_became_peered":"2023-06-23T18:05:04.130108+0000","last_unstale":"2023-06-23T18:05:05.438814+0000","last_undegraded":"2023-06-23T18:05:05.438814+0000","last_fullsized":"2023-06-23T18:05:05.438814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:36:14.870964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441514+0000","last_change":"2023-06-23T18:05:03.722805+0000","last_active":"2023-06-23T18:05:05.441514+0000","last_peered":"2023-06-23T18:05:05.441514+0000","last_clean":"2023-06-23T18:05:05.441514+0000","last_became_active":"2023-06-23T18:05:03.722448+0000","last_became_peered":"2023-06-23T18:05:03.722448+0000","last_unstale":"2023-06-23T18:05:05.441514+0000","last_undegraded":"2023-06-23T18:05:05.441514+0000","last_fullsized":"2023-06-23T18:05:05.441514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:41.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113091+0000","last_change":"2023-06-23T18:05:04.131677+0000","last_active":"2023-06-23T18:05:05.113091+0000","last_peered":"2023-06-23T18:05:05.113091+0000","last_clean":"2023-06-23T18:05:05.113091+0000","last_became_active":"2023-06-23T18:05:04.131405+0000","last_became_peered":"2023-06-23T18:05:04.131405+0000","last_unstale":"2023-06-23T18:05:05.113091+0000","last_undegraded":"2023-06-23T18:05:05.113091+0000","last_fullsized":"2023-06-23T18:05:05.113091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:02:44.968541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110284+0000","last_change":"2023-06-23T18:05:03.727582+0000","last_active":"2023-06-23T18:05:05.110284+0000","last_peered":"2023-06-23T18:05:05.110284+0000","last_clean":"2023-06-23T18:05:05.110284+0000","last_became_active":"2023-06-23T18:05:03.727119+0000","last_became_peered":"2023-06-23T18:05:03.727119+0000","last_unstale":"2023-06-23T18:05:05.110284+0000","last_undegraded":"2023-06-23T18:05:05.110284+0000","last_fullsized":"2023-06-23T18:05:05.110284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:27:27.895179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438698+0000","last_change":"2023-06-23T18:05:04.134695+0000","last_active":"2023-06-23T18:05:05.438698+0000","last_peered":"2023-06-23T18:05:05.438698+0000","last_clean":"2023-06-23T18:05:05.438698+0000","last_became_active":"2023-06-23T18:05:04.134276+0000","last_became_peered":"2023-06-23T18:05:04.134276+0000","last_unstale":"2023-06-23T18:05:05.438698+0000","last_undegraded":"2023-06-23T18:05:05.438698+0000","last_fullsized":"2023-06-23T18:05:05.438698+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:52:35.317430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113090+0000","last_change":"2023-06-23T18:05:03.722113+0000","last_active":"2023-06-23T18:05:05.113090+0000","last_peered":"2023-06-23T18:05:05.113090+0000","last_clean":"2023-06-23T18:05:05.113090+0000","last_became_active":"2023-06-23T18:05:03.721844+0000","last_became_peered":"2023-06-23T18:05:03.721844+0000","last_unstale":"2023-06-23T18:05:05.113090+0000","last_undegraded":"2023-06-23T18:05:05.113090+0000","last_fullsized":"2023-06-23T18:05:05.113090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:51:14.835648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440408+0000","last_change":"2023-06-23T18:05:04.130219+0000","last_active":"2023-06-23T18:05:05.440408+0000","last_peered":"2023-06-23T18:05:05.440408+0000","last_clean":"2023-06-23T18:05:05.440408+0000","last_became_active":"2023-06-23T18:05:04.129947+0000","last_became_peered":"2023-06-23T18:05:04.129947+0000","last_unstale":"2023-06-23T18:05:05.440408+0000","last_undegraded":"2023-06-23T18:05:05.440408+0000","last_fullsized":"2023-06-23T18:05:05.440408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:52:17.794961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438542+0000","last_change":"2023-06-23T18:05:03.728235+0000","last_active":"2023-06-23T18:05:05.438542+0000","last_peered":"2023-06-23T18:05:05.438542+0000","last_clean":"2023-06-23T18:05:05.438542+0000","last_became_active":"2023-06-23T18:05:03.727919+0000","last_became_peered":"2023-06-23T18:05:03.727919+0000","last_unstale":"2023-06-23T18:05:05.438542+0000","last_undegraded":"2023-06-23T18:05:05.438542+0000","last_fullsized":"2023-06-23T18:05:05.438542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:13.976421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439549+0000","last_change":"2023-06-23T18:05:04.124417+0000","last_active":"2023-06-23T18:05:05.439549+0000","last_peered":"2023-06-23T18:05:05.439549+0000","last_clean":"2023-06-23T18:05:05.439549+0000","last_became_active":"2023-06-23T18:05:04.124109+0000","last_became_peered":"2023-06-23T18:05:04.124109+0000","last_unstale":"2023-06-23T18:05:05.439549+0000","last_undegraded":"2023-06-23T18:05:05.439549+0000","last_fullsized":"2023-06-23T18:05:05.439549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:35:06.951667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440368+0000","last_change":"2023-06-23T18:05:03.727412+0000","last_active":"2023-06-23T18:05:05.440368+0000","last_peered":"2023-06-23T18:05:05.440368+0000","last_clean":"2023-06-23T18:05:05.440368+0000","last_became_active":"2023-06-23T18:05:03.727120+0000","last_became_peered":"2023-06-23T18:05:03.727120+0000","last_unstale":"2023-06-23T18:05:05.440368+0000","last_undegraded":"2023-06-23T18:05:05.440368+0000","last_fullsized":"2023-06-23T18:05:05.440368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113374+0000","last_change":"2023-06-23T18:05:04.137057+0000","last_active":"2023-06-23T18:05:05.113374+0000","last_peered":"2023-06-23T18:05:05.113374+0000","last_clean":"2023-06-23T18:05:05.113374+0000","last_became_active":"2023-06-23T18:05:04.136732+0000","last_became_peered":"2023-06-23T18:05:04.136732+0000","last_unstale":"2023-06-23T18:05:05.113374+0000","last_undegraded":"2023-06-23T18:05:05.113374+0000","last_fullsized":"2023-06-23T18:05:05.113374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:16:11.793941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113589+0000","last_change":"2023-06-23T18:05:03.729653+0000","last_active":"2023-06-23T18:05:05.113589+0000","last_peered":"2023-06-23T18:05:05.113589+0000","last_clean":"2023-06-23T18:05:05.113589+0000","last_became_active":"2023-06-23T18:05:03.729385+0000","last_became_peered":"2023-06-23T18:05:03.729385+0000","last_unstale":"2023-06-23T18:05:05.113589+0000","last_undegraded":"2023-06-23T18:05:05.113589+0000","last_fullsized":"2023-06-23T18:05:05.113589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:52:31.861434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441634+0000","last_change":"2023-06-23T18:05:04.123510+0000","last_active":"2023-06-23T18:05:05.441634+0000","last_peered":"2023-06-23T18:05:05.441634+0000","last_clean":"2023-06-23T18:05:05.441634+0000","last_became_active":"2023-06-23T18:05:04.123234+0000","last_became_peered":"2023-06-23T18:05:04.123234+0000","last_unstale":"2023-06-23T18:05:05.441634+0000","last_undegraded":"2023-06-23T18:05:05.441634+0000","last_fullsized":"2023-06-23T18:05:05.441634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:26.466042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441391+0000","last_change":"2023-06-23T18:05:03.728901+0000","last_active":"2023-06-23T18:05:05.441391+0000","last_peered":"2023-06-23T18:05:05.441391+0000","last_clean":"2023-06-23T18:05:05.441391+0000","last_became_active":"2023-06-23T18:05:03.728625+0000","last_became_peered":"2023-06-23T18:05:03.728625+0000","last_unstale":"2023-06-23T18:05:05.441391+0000","last_undegraded":"2023-06-23T18:05:05.441391+0000","last_fullsized":"2023-06-23T18:05:05.441391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:13:55.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110220+0000","last_change":"2023-06-23T18:05:04.126873+0000","last_active":"2023-06-23T18:05:05.110220+0000","last_peered":"2023-06-23T18:05:05.110220+0000","last_clean":"2023-06-23T18:05:05.110220+0000","last_became_active":"2023-06-23T18:05:04.126607+0000","last_became_peered":"2023-06-23T18:05:04.126607+0000","last_unstale":"2023-06-23T18:05:05.110220+0000","last_undegraded":"2023-06-23T18:05:05.110220+0000","last_fullsized":"2023-06-23T18:05:05.110220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:12:29.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441738+0000","last_change":"2023-06-23T18:05:03.725386+0000","last_active":"2023-06-23T18:05:05.441738+0000","last_peered":"2023-06-23T18:05:05.441738+0000","last_clean":"2023-06-23T18:05:05.441738+0000","last_became_active":"2023-06-23T18:05:03.725139+0000","last_became_peered":"2023-06-23T18:05:03.725139+0000","last_unstale":"2023-06-23T18:05:05.441738+0000","last_undegraded":"2023-06-23T18:05:05.441738+0000","last_fullsized":"2023-06-23T18:05:05.441738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:20.422452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441397+0000","last_change":"2023-06-23T18:05:04.126145+0000","last_active":"2023-06-23T18:05:05.441397+0000","last_peered":"2023-06-23T18:05:05.441397+0000","last_clean":"2023-06-23T18:05:05.441397+0000","last_became_active":"2023-06-23T18:05:04.125823+0000","last_became_peered":"2023-06-23T18:05:04.125823+0000","last_unstale":"2023-06-23T18:05:05.441397+0000","last_undegraded":"2023-06-23T18:05:05.441397+0000","last_fullsized":"2023-06-23T18:05:05.441397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:26.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439522+0000","last_change":"2023-06-23T18:05:03.721650+0000","last_active":"2023-06-23T18:05:05.439522+0000","last_peered":"2023-06-23T18:05:05.439522+0000","last_clean":"2023-06-23T18:05:05.439522+0000","last_became_active":"2023-06-23T18:05:03.721298+0000","last_became_peered":"2023-06-23T18:05:03.721298+0000","last_unstale":"2023-06-23T18:05:05.439522+0000","last_undegraded":"2023-06-23T18:05:05.439522+0000","last_fullsized":"2023-06-23T18:05:05.439522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:48:43.835290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439677+0000","last_change":"2023-06-23T18:05:04.129562+0000","last_active":"2023-06-23T18:05:05.439677+0000","last_peered":"2023-06-23T18:05:05.439677+0000","last_clean":"2023-06-23T18:05:05.439677+0000","last_became_active":"2023-06-23T18:05:04.129260+0000","last_became_peered":"2023-06-23T18:05:04.129260+0000","last_unstale":"2023-06-23T18:05:05.439677+0000","last_undegraded":"2023-06-23T18:05:05.439677+0000","last_fullsized":"2023-06-23T18:05:05.439677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:58:08.117888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110644+0000","last_change":"2023-06-23T18:05:03.729508+0000","last_active":"2023-06-23T18:05:05.110644+0000","last_peered":"2023-06-23T18:05:05.110644+0000","last_clean":"2023-06-23T18:05:05.110644+0000","last_became_active":"2023-06-23T18:05:03.729232+0000","last_became_peered":"2023-06-23T18:05:03.729232+0000","last_unstale":"2023-06-23T18:05:05.110644+0000","last_undegraded":"2023-06-23T18:05:05.110644+0000","last_fullsized":"2023-06-23T18:05:05.110644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439006+0000","last_change":"2023-06-23T18:05:04.119501+0000","last_active":"2023-06-23T18:05:05.439006+0000","last_peered":"2023-06-23T18:05:05.439006+0000","last_clean":"2023-06-23T18:05:05.439006+0000","last_became_active":"2023-06-23T18:05:04.119253+0000","last_became_peered":"2023-06-23T18:05:04.119253+0000","last_unstale":"2023-06-23T18:05:05.439006+0000","last_undegraded":"2023-06-23T18:05:05.439006+0000","last_fullsized":"2023-06-23T18:05:05.439006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:12:29.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441769+0000","last_change":"2023-06-23T18:05:03.729828+0000","last_active":"2023-06-23T18:05:05.441769+0000","last_peered":"2023-06-23T18:05:05.441769+0000","last_clean":"2023-06-23T18:05:05.441769+0000","last_became_active":"2023-06-23T18:05:03.729456+0000","last_became_peered":"2023-06-23T18:05:03.729456+0000","last_unstale":"2023-06-23T18:05:05.441769+0000","last_undegraded":"2023-06-23T18:05:05.441769+0000","last_fullsized":"2023-06-23T18:05:05.441769+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:08.542108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110021+0000","last_change":"2023-06-23T18:05:04.126754+0000","last_active":"2023-06-23T18:05:05.110021+0000","last_peered":"2023-06-23T18:05:05.110021+0000","last_clean":"2023-06-23T18:05:05.110021+0000","last_became_active":"2023-06-23T18:05:04.126409+0000","last_became_peered":"2023-06-23T18:05:04.126409+0000","last_unstale":"2023-06-23T18:05:05.110021+0000","last_undegraded":"2023-06-23T18:05:05.110021+0000","last_fullsized":"2023-06-23T18:05:05.110021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:18:35.359050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110004+0000","last_change":"2023-06-23T18:05:03.725690+0000","last_active":"2023-06-23T18:05:05.110004+0000","last_peered":"2023-06-23T18:05:05.110004+0000","last_clean":"2023-06-23T18:05:05.110004+0000","last_became_active":"2023-06-23T18:05:03.725416+0000","last_became_peered":"2023-06-23T18:05:03.725416+0000","last_unstale":"2023-06-23T18:05:05.110004+0000","last_undegraded":"2023-06-23T18:05:05.110004+0000","last_fullsized":"2023-06-23T18:05:05.110004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:39:17.933184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441932+0000","last_change":"2023-06-23T18:05:04.132003+0000","last_active":"2023-06-23T18:05:05.441932+0000","last_peered":"2023-06-23T18:05:05.441932+0000","last_clean":"2023-06-23T18:05:05.441932+0000","last_became_active":"2023-06-23T18:05:04.131694+0000","last_became_peered":"2023-06-23T18:05:04.131694+0000","last_unstale":"2023-06-23T18:05:05.441932+0000","last_undegraded":"2023-06-23T18:05:05.441932+0000","last_fullsized":"2023-06-23T18:05:05.441932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:26:05.299659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440521+0000","last_change":"2023-06-23T18:05:03.720382+0000","last_active":"2023-06-23T18:05:05.440521+0000","last_peered":"2023-06-23T18:05:05.440521+0000","last_clean":"2023-06-23T18:05:05.440521+0000","last_became_active":"2023-06-23T18:05:03.720090+0000","last_became_peered":"2023-06-23T18:05:03.720090+0000","last_unstale":"2023-06-23T18:05:05.440521+0000","last_undegraded":"2023-06-23T18:05:05.440521+0000","last_fullsized":"2023-06-23T18:05:05.440521+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:25.615986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438444+0000","last_change":"2023-06-23T18:05:04.121482+0000","last_active":"2023-06-23T18:05:05.438444+0000","last_peered":"2023-06-23T18:05:05.438444+0000","last_clean":"2023-06-23T18:05:05.438444+0000","last_became_active":"2023-06-23T18:05:04.121232+0000","last_became_peered":"2023-06-23T18:05:04.121232+0000","last_unstale":"2023-06-23T18:05:05.438444+0000","last_undegraded":"2023-06-23T18:05:05.438444+0000","last_fullsized":"2023-06-23T18:05:05.438444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:15.785497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438941+0000","last_change":"2023-06-23T18:05:03.725388+0000","last_active":"2023-06-23T18:05:05.438941+0000","last_peered":"2023-06-23T18:05:05.438941+0000","last_clean":"2023-06-23T18:05:05.438941+0000","last_became_active":"2023-06-23T18:05:03.725067+0000","last_became_peered":"2023-06-23T18:05:03.725067+0000","last_unstale":"2023-06-23T18:05:05.438941+0000","last_undegraded":"2023-06-23T18:05:05.438941+0000","last_fullsized":"2023-06-23T18:05:05.438941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:44:16.015011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439005+0000","last_change":"2023-06-23T18:05:04.127531+0000","last_active":"2023-06-23T18:05:05.439005+0000","last_peered":"2023-06-23T18:05:05.439005+0000","last_clean":"2023-06-23T18:05:05.439005+0000","last_became_active":"2023-06-23T18:05:04.127278+0000","last_became_peered":"2023-06-23T18:05:04.127278+0000","last_unstale":"2023-06-23T18:05:05.439005+0000","last_undegraded":"2023-06-23T18:05:05.439005+0000","last_fullsized":"2023-06-23T18:05:05.439005+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:17:03.193671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113539+0000","last_change":"2023-06-23T18:05:03.727766+0000","last_active":"2023-06-23T18:05:05.113539+0000","last_peered":"2023-06-23T18:05:05.113539+0000","last_clean":"2023-06-23T18:05:05.113539+0000","last_became_active":"2023-06-23T18:05:03.727121+0000","last_became_peered":"2023-06-23T18:05:03.727121+0000","last_unstale":"2023-06-23T18:05:05.113539+0000","last_undegraded":"2023-06-23T18:05:05.113539+0000","last_fullsized":"2023-06-23T18:05:05.113539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:52.582036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439224+0000","last_change":"2023-06-23T18:05:04.127883+0000","last_active":"2023-06-23T18:05:05.439224+0000","last_peered":"2023-06-23T18:05:05.439224+0000","last_clean":"2023-06-23T18:05:05.439224+0000","last_became_active":"2023-06-23T18:05:04.127597+0000","last_became_peered":"2023-06-23T18:05:04.127597+0000","last_unstale":"2023-06-23T18:05:05.439224+0000","last_undegraded":"2023-06-23T18:05:05.439224+0000","last_fullsized":"2023-06-23T18:05:05.439224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:44:26.837150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440302+0000","last_change":"2023-06-23T18:05:03.731975+0000","last_active":"2023-06-23T18:05:05.440302+0000","last_peered":"2023-06-23T18:05:05.440302+0000","last_clean":"2023-06-23T18:05:05.440302+0000","last_became_active":"2023-06-23T18:05:03.731696+0000","last_became_peered":"2023-06-23T18:05:03.731696+0000","last_unstale":"2023-06-23T18:05:05.440302+0000","last_undegraded":"2023-06-23T18:05:05.440302+0000","last_fullsized":"2023-06-23T18:05:05.440302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:25.462178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439527+0000","last_change":"2023-06-23T18:05:04.131040+0000","last_active":"2023-06-23T18:05:05.439527+0000","last_peered":"2023-06-23T18:05:05.439527+0000","last_clean":"2023-06-23T18:05:05.439527+0000","last_became_active":"2023-06-23T18:05:04.130585+0000","last_became_peered":"2023-06-23T18:05:04.130585+0000","last_unstale":"2023-06-23T18:05:05.439527+0000","last_undegraded":"2023-06-23T18:05:05.439527+0000","last_fullsized":"2023-06-23T18:05:05.439527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:53:12.390982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110265+0000","last_change":"2023-06-23T18:05:03.725776+0000","last_active":"2023-06-23T18:05:05.110265+0000","last_peered":"2023-06-23T18:05:05.110265+0000","last_clean":"2023-06-23T18:05:05.110265+0000","last_became_active":"2023-06-23T18:05:03.725254+0000","last_became_peered":"2023-06-23T18:05:03.725254+0000","last_unstale":"2023-06-23T18:05:05.110265+0000","last_undegraded":"2023-06-23T18:05:05.110265+0000","last_fullsized":"2023-06-23T18:05:05.110265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:45:23.912814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110047+0000","last_change":"2023-06-23T18:05:04.127175+0000","last_active":"2023-06-23T18:05:05.110047+0000","last_peered":"2023-06-23T18:05:05.110047+0000","last_clean":"2023-06-23T18:05:05.110047+0000","last_became_active":"2023-06-23T18:05:04.126875+0000","last_became_peered":"2023-06-23T18:05:04.126875+0000","last_unstale":"2023-06-23T18:05:05.110047+0000","last_undegraded":"2023-06-23T18:05:05.110047+0000","last_fullsized":"2023-06-23T18:05:05.110047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:23:43.253102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110558+0000","last_change":"2023-06-23T18:05:03.727398+0000","last_active":"2023-06-23T18:05:05.110558+0000","last_peered":"2023-06-23T18:05:05.110558+0000","last_clean":"2023-06-23T18:05:05.110558+0000","last_became_active":"2023-06-23T18:05:03.726739+0000","last_became_peered":"2023-06-23T18:05:03.726739+0000","last_unstale":"2023-06-23T18:05:05.110558+0000","last_undegraded":"2023-06-23T18:05:05.110558+0000","last_fullsized":"2023-06-23T18:05:05.110558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:13.976421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438843+0000","last_change":"2023-06-23T18:05:04.122243+0000","last_active":"2023-06-23T18:05:05.438843+0000","last_peered":"2023-06-23T18:05:05.438843+0000","last_clean":"2023-06-23T18:05:05.438843+0000","last_became_active":"2023-06-23T18:05:04.121991+0000","last_became_peered":"2023-06-23T18:05:04.121991+0000","last_unstale":"2023-06-23T18:05:05.438843+0000","last_undegraded":"2023-06-23T18:05:05.438843+0000","last_fullsized":"2023-06-23T18:05:05.438843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:18:35.359050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438930+0000","last_change":"2023-06-23T18:05:03.723358+0000","last_active":"2023-06-23T18:05:05.438930+0000","last_peered":"2023-06-23T18:05:05.438930+0000","last_clean":"2023-06-23T18:05:05.438930+0000","last_became_active":"2023-06-23T18:05:03.723005+0000","last_became_peered":"2023-06-23T18:05:03.723005+0000","last_unstale":"2023-06-23T18:05:05.438930+0000","last_undegraded":"2023-06-23T18:05:05.438930+0000","last_fullsized":"2023-06-23T18:05:05.438930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:09.824569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437747+0000","last_change":"2023-06-23T18:05:04.122156+0000","last_active":"2023-06-23T18:05:05.437747+0000","last_peered":"2023-06-23T18:05:05.437747+0000","last_clean":"2023-06-23T18:05:05.437747+0000","last_became_active":"2023-06-23T18:05:04.121872+0000","last_became_peered":"2023-06-23T18:05:04.121872+0000","last_unstale":"2023-06-23T18:05:05.437747+0000","last_undegraded":"2023-06-23T18:05:05.437747+0000","last_fullsized":"2023-06-23T18:05:05.437747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:01:39.428071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441413+0000","last_change":"2023-06-23T18:05:03.727740+0000","last_active":"2023-06-23T18:05:05.441413+0000","last_peered":"2023-06-23T18:05:05.441413+0000","last_clean":"2023-06-23T18:05:05.441413+0000","last_became_active":"2023-06-23T18:05:03.727377+0000","last_became_peered":"2023-06-23T18:05:03.727377+0000","last_unstale":"2023-06-23T18:05:05.441413+0000","last_undegraded":"2023-06-23T18:05:05.441413+0000","last_fullsized":"2023-06-23T18:05:05.441413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:55:20.422452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441407+0000","last_change":"2023-06-23T18:05:04.120326+0000","last_active":"2023-06-23T18:05:05.441407+0000","last_peered":"2023-06-23T18:05:05.441407+0000","last_clean":"2023-06-23T18:05:05.441407+0000","last_became_active":"2023-06-23T18:05:04.120027+0000","last_became_peered":"2023-06-23T18:05:04.120027+0000","last_unstale":"2023-06-23T18:05:05.441407+0000","last_undegraded":"2023-06-23T18:05:05.441407+0000","last_fullsized":"2023-06-23T18:05:05.441407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:43:26.466042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.442036+0000","last_change":"2023-06-23T18:05:03.729752+0000","last_active":"2023-06-23T18:05:05.442036+0000","last_peered":"2023-06-23T18:05:05.442036+0000","last_clean":"2023-06-23T18:05:05.442036+0000","last_became_active":"2023-06-23T18:05:03.729439+0000","last_became_peered":"2023-06-23T18:05:03.729439+0000","last_unstale":"2023-06-23T18:05:05.442036+0000","last_undegraded":"2023-06-23T18:05:05.442036+0000","last_fullsized":"2023-06-23T18:05:05.442036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:02:46.398537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110188+0000","last_change":"2023-06-23T18:05:04.127646+0000","last_active":"2023-06-23T18:05:05.110188+0000","last_peered":"2023-06-23T18:05:05.110188+0000","last_clean":"2023-06-23T18:05:05.110188+0000","last_became_active":"2023-06-23T18:05:04.127167+0000","last_became_peered":"2023-06-23T18:05:04.127167+0000","last_unstale":"2023-06-23T18:05:05.110188+0000","last_undegraded":"2023-06-23T18:05:05.110188+0000","last_fullsized":"2023-06-23T18:05:05.110188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:52:35.317430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.442009+0000","last_change":"2023-06-23T18:05:03.724361+0000","last_active":"2023-06-23T18:05:05.442009+0000","last_peered":"2023-06-23T18:05:05.442009+0000","last_clean":"2023-06-23T18:05:05.442009+0000","last_became_active":"2023-06-23T18:05:03.724064+0000","last_became_peered":"2023-06-23T18:05:03.724064+0000","last_unstale":"2023-06-23T18:05:05.442009+0000","last_undegraded":"2023-06-23T18:05:05.442009+0000","last_fullsized":"2023-06-23T18:05:05.442009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:29:49.123036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439844+0000","last_change":"2023-06-23T18:03:33.248935+0000","last_active":"2023-06-23T18:05:05.439844+0000","last_peered":"2023-06-23T18:05:05.439844+0000","last_clean":"2023-06-23T18:05:05.439844+0000","last_became_active":"2023-06-23T18:03:33.248522+0000","last_became_peered":"2023-06-23T18:03:33.248522+0000","last_unstale":"2023-06-23T18:05:05.439844+0000","last_undegraded":"2023-06-23T18:05:05.439844+0000","last_fullsized":"2023-06-23T18:05:05.439844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:27:47.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110335+0000","last_change":"2023-06-23T18:05:04.131076+0000","last_active":"2023-06-23T18:05:05.110335+0000","last_peered":"2023-06-23T18:05:05.110335+0000","last_clean":"2023-06-23T18:05:05.110335+0000","last_became_active":"2023-06-23T18:05:04.126407+0000","last_became_peered":"2023-06-23T18:05:04.126407+0000","last_unstale":"2023-06-23T18:05:05.110335+0000","last_undegraded":"2023-06-23T18:05:05.110335+0000","last_fullsized":"2023-06-23T18:05:05.110335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:15.534113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.113266+0000","last_change":"2023-06-23T18:05:03.727717+0000","last_active":"2023-06-23T18:05:05.113266+0000","last_peered":"2023-06-23T18:05:05.113266+0000","last_clean":"2023-06-23T18:05:05.113266+0000","last_became_active":"2023-06-23T18:05:03.727394+0000","last_became_peered":"2023-06-23T18:05:03.727394+0000","last_unstale":"2023-06-23T18:05:05.113266+0000","last_undegraded":"2023-06-23T18:05:05.113266+0000","last_fullsized":"2023-06-23T18:05:05.113266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:58:41.573936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439185+0000","last_change":"2023-06-23T18:03:33.248220+0000","last_active":"2023-06-23T18:05:05.439185+0000","last_peered":"2023-06-23T18:05:05.439185+0000","last_clean":"2023-06-23T18:05:05.439185+0000","last_became_active":"2023-06-23T18:03:33.247890+0000","last_became_peered":"2023-06-23T18:03:33.247890+0000","last_unstale":"2023-06-23T18:05:05.439185+0000","last_undegraded":"2023-06-23T18:05:05.439185+0000","last_fullsized":"2023-06-23T18:05:05.439185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:37:47.464339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438096+0000","last_change":"2023-06-23T18:05:04.129424+0000","last_active":"2023-06-23T18:05:05.438096+0000","last_peered":"2023-06-23T18:05:05.438096+0000","last_clean":"2023-06-23T18:05:05.438096+0000","last_became_active":"2023-06-23T18:05:04.128128+0000","last_became_peered":"2023-06-23T18:05:04.128128+0000","last_unstale":"2023-06-23T18:05:05.438096+0000","last_undegraded":"2023-06-23T18:05:05.438096+0000","last_fullsized":"2023-06-23T18:05:05.438096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:42.099943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439282+0000","last_change":"2023-06-23T18:05:03.726337+0000","last_active":"2023-06-23T18:05:05.439282+0000","last_peered":"2023-06-23T18:05:05.439282+0000","last_clean":"2023-06-23T18:05:05.439282+0000","last_became_active":"2023-06-23T18:05:03.726012+0000","last_became_peered":"2023-06-23T18:05:03.726012+0000","last_unstale":"2023-06-23T18:05:05.439282+0000","last_undegraded":"2023-06-23T18:05:05.439282+0000","last_fullsized":"2023-06-23T18:05:05.439282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:41:14.934399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441985+0000","last_change":"2023-06-23T18:03:33.324347+0000","last_active":"2023-06-23T18:05:05.441985+0000","last_peered":"2023-06-23T18:05:05.441985+0000","last_clean":"2023-06-23T18:05:05.441985+0000","last_became_active":"2023-06-23T18:03:33.323965+0000","last_became_peered":"2023-06-23T18:03:33.323965+0000","last_unstale":"2023-06-23T18:05:05.441985+0000","last_undegraded":"2023-06-23T18:05:05.441985+0000","last_fullsized":"2023-06-23T18:05:05.441985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:50:09.384203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.112886+0000","last_change":"2023-06-23T18:05:04.125214+0000","last_active":"2023-06-23T18:05:05.112886+0000","last_peered":"2023-06-23T18:05:05.112886+0000","last_clean":"2023-06-23T18:05:05.112886+0000","last_became_active":"2023-06-23T18:05:04.124802+0000","last_became_peered":"2023-06-23T18:05:04.124802+0000","last_unstale":"2023-06-23T18:05:05.112886+0000","last_undegraded":"2023-06-23T18:05:05.112886+0000","last_fullsized":"2023-06-23T18:05:05.112886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:29:25.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440728+0000","last_change":"2023-06-23T18:05:03.727367+0000","last_active":"2023-06-23T18:05:05.440728+0000","last_peered":"2023-06-23T18:05:05.440728+0000","last_clean":"2023-06-23T18:05:05.440728+0000","last_became_active":"2023-06-23T18:05:03.726518+0000","last_became_peered":"2023-06-23T18:05:03.726518+0000","last_unstale":"2023-06-23T18:05:05.440728+0000","last_undegraded":"2023-06-23T18:05:05.440728+0000","last_fullsized":"2023-06-23T18:05:05.440728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:41.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438233+0000","last_change":"2023-06-23T18:03:33.321641+0000","last_active":"2023-06-23T18:05:05.438233+0000","last_peered":"2023-06-23T18:05:05.438233+0000","last_clean":"2023-06-23T18:05:05.438233+0000","last_became_active":"2023-06-23T18:03:33.321288+0000","last_became_peered":"2023-06-23T18:03:33.321288+0000","last_unstale":"2023-06-23T18:05:05.438233+0000","last_undegraded":"2023-06-23T18:05:05.438233+0000","last_fullsized":"2023-06-23T18:05:05.438233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:29:07.778826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439662+0000","last_change":"2023-06-23T18:05:04.129560+0000","last_active":"2023-06-23T18:05:05.439662+0000","last_peered":"2023-06-23T18:05:05.439662+0000","last_clean":"2023-06-23T18:05:05.439662+0000","last_became_active":"2023-06-23T18:05:04.129254+0000","last_became_peered":"2023-06-23T18:05:04.129254+0000","last_unstale":"2023-06-23T18:05:05.439662+0000","last_undegraded":"2023-06-23T18:05:05.439662+0000","last_fullsized":"2023-06-23T18:05:05.439662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:10:35.937401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.110513+0000","last_change":"2023-06-23T18:05:03.732330+0000","last_active":"2023-06-23T18:05:05.110513+0000","last_peered":"2023-06-23T18:05:05.110513+0000","last_clean":"2023-06-23T18:05:05.110513+0000","last_became_active":"2023-06-23T18:05:03.732034+0000","last_became_peered":"2023-06-23T18:05:03.732034+0000","last_unstale":"2023-06-23T18:05:05.110513+0000","last_undegraded":"2023-06-23T18:05:05.110513+0000","last_fullsized":"2023-06-23T18:05:05.110513+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:34:53.415660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439127+0000","last_change":"2023-06-23T18:03:33.248978+0000","last_active":"2023-06-23T18:05:05.439127+0000","last_peered":"2023-06-23T18:05:05.439127+0000","last_clean":"2023-06-23T18:05:05.439127+0000","last_became_active":"2023-06-23T18:03:33.248707+0000","last_became_peered":"2023-06-23T18:03:33.248707+0000","last_unstale":"2023-06-23T18:05:05.439127+0000","last_undegraded":"2023-06-23T18:05:05.439127+0000","last_fullsized":"2023-06-23T18:05:05.439127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:50:46.322252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440130+0000","last_change":"2023-06-23T18:05:04.118390+0000","last_active":"2023-06-23T18:05:05.440130+0000","last_peered":"2023-06-23T18:05:05.440130+0000","last_clean":"2023-06-23T18:05:05.440130+0000","last_became_active":"2023-06-23T18:05:04.118051+0000","last_became_peered":"2023-06-23T18:05:04.118051+0000","last_unstale":"2023-06-23T18:05:05.440130+0000","last_undegraded":"2023-06-23T18:05:05.440130+0000","last_fullsized":"2023-06-23T18:05:05.440130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:42.099943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441882+0000","last_change":"2023-06-23T18:05:03.724378+0000","last_active":"2023-06-23T18:05:05.441882+0000","last_peered":"2023-06-23T18:05:05.441882+0000","last_clean":"2023-06-23T18:05:05.441882+0000","last_became_active":"2023-06-23T18:05:03.724073+0000","last_became_peered":"2023-06-23T18:05:03.724073+0000","last_unstale":"2023-06-23T18:05:05.441882+0000","last_undegraded":"2023-06-23T18:05:05.441882+0000","last_fullsized":"2023-06-23T18:05:05.441882+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:11:20.986246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.437278+0000","last_change":"2023-06-23T18:03:33.248131+0000","last_active":"2023-06-23T18:05:05.437278+0000","last_peered":"2023-06-23T18:05:05.437278+0000","last_clean":"2023-06-23T18:05:05.437278+0000","last_became_active":"2023-06-23T18:03:33.247789+0000","last_became_peered":"2023-06-23T18:03:33.247789+0000","last_unstale":"2023-06-23T18:05:05.437278+0000","last_undegraded":"2023-06-23T18:05:05.437278+0000","last_fullsized":"2023-06-23T18:05:05.437278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:12:57.127001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441600+0000","last_change":"2023-06-23T18:05:04.119695+0000","last_active":"2023-06-23T18:05:05.441600+0000","last_peered":"2023-06-23T18:05:05.441600+0000","last_clean":"2023-06-23T18:05:05.441600+0000","last_became_active":"2023-06-23T18:05:04.119404+0000","last_became_peered":"2023-06-23T18:05:04.119404+0000","last_unstale":"2023-06-23T18:05:05.441600+0000","last_undegraded":"2023-06-23T18:05:05.441600+0000","last_fullsized":"2023-06-23T18:05:05.441600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.438019+0000","last_change":"2023-06-23T18:05:03.726382+0000","last_active":"2023-06-23T18:05:05.438019+0000","last_peered":"2023-06-23T18:05:05.438019+0000","last_clean":"2023-06-23T18:05:05.438019+0000","last_became_active":"2023-06-23T18:05:03.725770+0000","last_became_peered":"2023-06-23T18:05:03.725770+0000","last_unstale":"2023-06-23T18:05:05.438019+0000","last_undegraded":"2023-06-23T18:05:05.438019+0000","last_fullsized":"2023-06-23T18:05:05.438019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:56:28.102729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440955+0000","last_change":"2023-06-23T18:03:33.249196+0000","last_active":"2023-06-23T18:05:05.440955+0000","last_peered":"2023-06-23T18:05:05.440955+0000","last_clean":"2023-06-23T18:05:05.440955+0000","last_became_active":"2023-06-23T18:03:33.248760+0000","last_became_peered":"2023-06-23T18:03:33.248760+0000","last_unstale":"2023-06-23T18:05:05.440955+0000","last_undegraded":"2023-06-23T18:05:05.440955+0000","last_fullsized":"2023-06-23T18:05:05.440955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:57:39.355724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441109+0000","last_change":"2023-06-23T18:05:04.129350+0000","last_active":"2023-06-23T18:05:05.441109+0000","last_peered":"2023-06-23T18:05:05.441109+0000","last_clean":"2023-06-23T18:05:05.441109+0000","last_became_active":"2023-06-23T18:05:04.127874+0000","last_became_peered":"2023-06-23T18:05:04.127874+0000","last_unstale":"2023-06-23T18:05:05.441109+0000","last_undegraded":"2023-06-23T18:05:05.441109+0000","last_fullsized":"2023-06-23T18:05:05.441109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:03.694282+0000","last_clean_scrub_stamp":"2023-06-23T18:05:03.694282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:20.540183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.440697+0000","last_change":"2023-06-23T18:05:03.726812+0000","last_active":"2023-06-23T18:05:05.440697+0000","last_peered":"2023-06-23T18:05:05.440697+0000","last_clean":"2023-06-23T18:05:05.440697+0000","last_became_active":"2023-06-23T18:05:03.726544+0000","last_became_peered":"2023-06-23T18:05:03.726544+0000","last_unstale":"2023-06-23T18:05:05.440697+0000","last_undegraded":"2023-06-23T18:05:05.440697+0000","last_fullsized":"2023-06-23T18:05:05.440697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:05:02.690418+0000","last_clean_scrub_stamp":"2023-06-23T18:05:02.690418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:08.542108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.439455+0000","last_change":"2023-06-23T18:03:33.249361+0000","last_active":"2023-06-23T18:05:05.439455+0000","last_peered":"2023-06-23T18:05:05.439455+0000","last_clean":"2023-06-23T18:05:05.439455+0000","last_became_active":"2023-06-23T18:03:33.248687+0000","last_became_peered":"2023-06-23T18:03:33.248687+0000","last_unstale":"2023-06-23T18:05:05.439455+0000","last_undegraded":"2023-06-23T18:05:05.439455+0000","last_fullsized":"2023-06-23T18:05:05.439455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:11.402900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-23T18:05:05.441122+0000","last_change":"2023-06-23T18:03:33.325434+0000","last_active":"2023-06-23T18:05:05.441122+0000","last_peered":"2023-06-23T18:05:05.441122+0000","last_clean":"2023-06-23T18:05:05.441122+0000","last_became_active":"2023-06-23T18:03:33.324083+0000","last_became_peered":"2023-06-23T18:03:33.324083+0000","last_unstale":"2023-06-23T18:05:05.441122+0000","last_undegraded":"2023-06-23T18:05:05.441122+0000","last_fullsized":"2023-06-23T18:05:05.441122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:03:32.221574+0000","last_clean_scrub_stamp":"2023-06-23T18:03:32.221574+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:50:46.322252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":1048576,"data_stored":794055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.0069999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.175},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0389999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":458752,"data_stored":204269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0129999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.96199999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.093}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.98299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":454656,"data_stored":203667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":1.038}]},{"osd":2,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.871}]},{"osd":3,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.76200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.099},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.1459999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0169999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":462848,"data_stored":205161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.518},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.556},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.6259999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.3180000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.145},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.673}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":462848,"data_stored":204235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.629}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.246}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1559999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31504,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340336,"statfs":{"total":96636764160,"available":96604504064,"internally_reserved":0,"allocated":466944,"data_stored":205183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78400000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.94899999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.079}]},{"osd":4,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.89400000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.264},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.1220000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.024},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:04:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.218},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31492,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340348,"statfs":{"total":96636764160,"available":96604516352,"internally_reserved":0,"allocated":454656,"data_stored":203667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.502}]},{"osd":2,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:04:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1048576,"data_stored":793999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.216}]},{"osd":3,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.333},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.117},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.022}]},{"osd":6,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.159},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.091}]},{"osd":7,"last update":"Fri Jun 23 18:04:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.29},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.258}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:05:08.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-23T18:05:08.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-06-23T18:05:08.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.907+0000 7f3b0b59e700 1 Processor -- start 2023-06-23T18:05:08.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.907+0000 7f3b0b59e700 1 -- start start 2023-06-23T18:05:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b0c158b20 0x7f3b0c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c06eab0 con 0x7f3b0c070740 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0cdf20 con 0x7f3b0c158b20 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0cd680 con 0x7f3b0c158ef0 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.22:50328/0 (socket says 172.21.15.22:50328) 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0a59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b0c158b20 0x7f3b0c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 -- 172.21.15.22:0/302998629 learned_addr learned my addr 172.21.15.22:0/302998629 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 -- 172.21.15.22:0/302998629 >> v2:172.21.15.169:3300/0 conn(0x7f3b0c158b20 msgr2=0x7f3b0c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 --2- 172.21.15.22:0/302998629 >> v2:172.21.15.169:3300/0 conn(0x7f3b0c158b20 0x7f3b0c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 -- 172.21.15.22:0/302998629 >> v1:172.21.15.169:6789/0 conn(0x7f3b0c070740 legacy=0x7f3b0c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 -- 172.21.15.22:0/302998629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0cd960 con 0x7f3b0c158ef0 2023-06-23T18:05:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b03fff700 1 --2- 172.21.15.22:0/302998629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009910 tx=0x7f3af4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e532dc9f48d474 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b09d9b700 1 -- 172.21.15.22:0/302998629 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4011830 con 0x7f3b0c158ef0 2023-06-23T18:05:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b09d9b700 1 -- 172.21.15.22:0/302998629 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af4011990 con 0x7f3b0c158ef0 2023-06-23T18:05:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b09d9b700 1 -- 172.21.15.22:0/302998629 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4011d40 con 0x7f3b0c158ef0 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- 172.21.15.22:0/302998629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 msgr2=0x7f3b0c0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/302998629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009910 tx=0x7f3af4009c20 comp rx=0 tx=0).stop 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- 172.21.15.22:0/302998629 shutdown_connections 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/302998629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158ef0 0x7f3b0c0b1780 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/302998629 >> v2:172.21.15.169:3300/0 conn(0x7f3b0c158b20 0x7f3b0c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- 172.21.15.22:0/302998629 >> 172.21.15.22:0/302998629 conn(0x7f3b0c06b700 msgr2=0x7f3b0c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:08.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- 172.21.15.22:0/302998629 shutdown_connections 2023-06-23T18:05:08.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.911+0000 7f3b0b59e700 1 -- 172.21.15.22:0/302998629 wait complete. 2023-06-23T18:05:08.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.915+0000 7f3b0b59e700 1 Processor -- start 2023-06-23T18:05:08.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.915+0000 7f3b0b59e700 1 -- start start 2023-06-23T18:05:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b0c070740 0x7f3b0c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 0x7f3b0c14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b1f70 con 0x7f3b0c158ef0 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b2810 con 0x7f3b0c070740 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c8df0 con 0x7f3b0c158b20 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 0x7f3b0c14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44576/0 (socket says 172.21.15.22:44576) 2023-06-23T18:05:08.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 -- 172.21.15.22:0/2387408751 learned_addr learned my addr 172.21.15.22:0/2387408751 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:08.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0a59c700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b0c070740 0x7f3b0c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 -- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 msgr2=0x7f3b0c14bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 0x7f3b0c14bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 -- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b0c070740 msgr2=0x7f3b0c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b0c070740 0x7f3b0c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 -- 172.21.15.22:0/2387408751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4009570 con 0x7f3b0c158ef0 2023-06-23T18:05:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b03fff700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 0x7f3b0c14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0ad9d700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400cc50 tx=0x7f3b0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04003c20 con 0x7f3b0c158ef0 2023-06-23T18:05:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04003d80 con 0x7f3b0c158ef0 2023-06-23T18:05:08.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c0b21c0 con 0x7f3b0c158ef0 2023-06-23T18:05:08.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400aab0 con 0x7f3b0c158ef0 2023-06-23T18:05:08.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.919+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c0c8ae0 con 0x7f3b0c158ef0 2023-06-23T18:05:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f3b0400ac10 con 0x7f3b0c158ef0 2023-06-23T18:05:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c04e4c0 con 0x7f3b0c158ef0 2023-06-23T18:05:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b02ffd700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 0x7f3aec0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3b04083d80 con 0x7f3b0c158ef0 2023-06-23T18:05:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b0a59c700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 0x7f3aec0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.923+0000 7f3b0a59c700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 0x7f3aec0687a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009bf0 tx=0x7f3afc009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:08.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:08.935+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f3b04051150 con 0x7f3b0c158ef0 2023-06-23T18:05:09.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.155+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3b0c0b2950 con 0x7f3b0c158ef0 2023-06-23T18:05:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.167+0000 7f3b02ffd700 1 -- 172.21.15.22:0/2387408751 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3b04054770 con 0x7f3b0c158ef0 2023-06-23T18:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2023-06-23T18:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBK3pVksM9cBxAAxr2TtBTZIBuP1bjAu3SXFA== 2023-06-23T18:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2023-06-23T18:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2023-06-23T18:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-06-23T18:05:09.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 msgr2=0x7f3aec0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 0x7f3aec0687a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009bf0 tx=0x7f3afc009240 comp rx=0 tx=0).stop 2023-06-23T18:05:09.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 msgr2=0x7f3b0c157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400cc50 tx=0x7f3b0400cf60 comp rx=0 tx=0).stop 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 shutdown_connections 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b0c158ef0 0x7f3b0c157490 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b0c158b20 0x7f3b0c14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f3aec0662f0 0x7f3aec0687a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 --2- 172.21.15.22:0/2387408751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b0c070740 0x7f3b0c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 >> 172.21.15.22:0/2387408751 conn(0x7f3b0c06b700 msgr2=0x7f3b0c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 shutdown_connections 2023-06-23T18:05:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.171+0000 7f3b0b59e700 1 -- 172.21.15.22:0/2387408751 wait complete. 2023-06-23T18:05:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2023-06-23T18:05:09.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:05:09.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.303+0000 7fac648c0700 1 Processor -- start 2023-06-23T18:05:09.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.303+0000 7fac648c0700 1 -- start start 2023-06-23T18:05:09.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 0x7fac600ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 0x7fac6015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6015a080 con 0x7fac600ce780 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac600b62e0 con 0x7fac600cde70 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600cc830 con 0x7fac600cf1b0 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5f59e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fac600ce780 0x7fac6015dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34590/0 (socket says 172.21.15.22:34590) 2023-06-23T18:05:09.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5f59e700 1 -- 172.21.15.22:0/1324292356 learned_addr learned my addr 172.21.15.22:0/1324292356 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:09.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1324292356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 0x7fac6015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5ed9d700 1 --2- 172.21.15.22:0/1324292356 >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 0x7fac600ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5dd9b700 1 -- 172.21.15.22:0/1324292356 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358207416 0 0) 0x7fac58001490 con 0x7fac600ce780 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5dd9b700 1 -- 172.21.15.22:0/1324292356 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac4c0035d0 con 0x7fac600ce780 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 -- 172.21.15.22:0/1324292356 >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 msgr2=0x7fac600ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1324292356 >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 0x7fac600ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 -- 172.21.15.22:0/1324292356 >> v1:172.21.15.22:6789/0 conn(0x7fac600ce780 legacy=0x7fac6015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 -- 172.21.15.22:0/1324292356 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600cd3b0 con 0x7fac600cf1b0 2023-06-23T18:05:09.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5ed9d700 1 --2- 172.21.15.22:0/1324292356 >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 0x7fac600ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:09.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1324292356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 0x7fac6015d7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fac48009850 tx=0x7fac48009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42283da3372d8493 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:09.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5dd9b700 1 -- 172.21.15.22:0/1324292356 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48011830 con 0x7fac600cf1b0 2023-06-23T18:05:09.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5dd9b700 1 -- 172.21.15.22:0/1324292356 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48011990 con 0x7fac600cf1b0 2023-06-23T18:05:09.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac5dd9b700 1 -- 172.21.15.22:0/1324292356 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48011d00 con 0x7fac600cf1b0 2023-06-23T18:05:09.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- 172.21.15.22:0/1324292356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 msgr2=0x7fac6015d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 --2- 172.21.15.22:0/1324292356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 0x7fac6015d7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fac48009850 tx=0x7fac48009b60 comp rx=0 tx=0).stop 2023-06-23T18:05:09.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- 172.21.15.22:0/1324292356 shutdown_connections 2023-06-23T18:05:09.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 --2- 172.21.15.22:0/1324292356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600cf1b0 0x7fac6015d7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 --2- 172.21.15.22:0/1324292356 >> v2:172.21.15.169:3300/0 conn(0x7fac600cde70 0x7fac600ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- 172.21.15.22:0/1324292356 >> 172.21.15.22:0/1324292356 conn(0x7fac60069960 msgr2=0x7fac60069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:09.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- 172.21.15.22:0/1324292356 shutdown_connections 2023-06-23T18:05:09.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.307+0000 7fac648c0700 1 -- 172.21.15.22:0/1324292356 wait complete. 2023-06-23T18:05:09.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 Processor -- start 2023-06-23T18:05:09.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 -- start start 2023-06-23T18:05:09.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600ce780 0x7fac601eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 0x7fac600a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac600a8620 con 0x7fac600cde70 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac601ef320 con 0x7fac600cf1b0 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac648c0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600a7aa0 con 0x7fac600ce780 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac5ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac5ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44578/0 (socket says 172.21.15.22:44578) 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.311+0000 7fac5ed9d700 1 -- 172.21.15.22:0/1770791406 learned_addr learned my addr 172.21.15.22:0/1770791406 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:09.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600ce780 0x7fac601eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5f59e700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 0x7fac600a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 -- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600ce780 msgr2=0x7fac601eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600ce780 0x7fac601eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 -- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 msgr2=0x7fac600a89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 0x7fac600a89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 -- 172.21.15.22:0/1770791406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac500094b0 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5f59e700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 0x7fac600a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac5ed9d700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ec10 tx=0x7fac50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000c8e0 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5000ca40 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac480094b0 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000cdd0 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac600a8060 con 0x7fac600cde70 2023-06-23T18:05:09.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6004e4c0 con 0x7fac600cde70 2023-06-23T18:05:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fac50013470 con 0x7fac600cde70 2023-06-23T18:05:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.315+0000 7fac577fe700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 0x7fac400687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.319+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 0x7fac400687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.319+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac50083fa0 con 0x7fac600cde70 2023-06-23T18:05:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.319+0000 7fac5e59c700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 0x7fac400687a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac480090b0 tx=0x7fac48008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:09.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.327+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fac500b4700 con 0x7fac600cde70 2023-06-23T18:05:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.527+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac600a87a0 con 0x7fac600cde70 2023-06-23T18:05:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac577fe700 1 -- 172.21.15.22:0/1770791406 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fac50017090 con 0x7fac600cde70 2023-06-23T18:05:09.540 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:09.541 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1697006502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6835","nonce":1697006502},{"type":"v1","addr":"172.21.15.22:6837","nonce":1697006502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2478537796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2478537796},{"type":"v1","addr":"172.21.15.169:6834","nonce":2478537796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/880910702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":880910702},{"type":"v1","addr":"172.21.15.169:6835","nonce":880910702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-23T18:05:04.092062+0000","modified":"2023-06-23T18:05:07.368660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6836/3366896912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3366896912},{"type":"v1","addr":"172.21.15.22:6836","nonce":3366896912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-23T18:05:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 msgr2=0x7fac400687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 0x7fac400687a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac480090b0 tx=0x7fac48008030 comp rx=0 tx=0).stop 2023-06-23T18:05:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 msgr2=0x7fac6015c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ec10 tx=0x7fac50009e30 comp rx=0 tx=0).stop 2023-06-23T18:05:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 shutdown_connections 2023-06-23T18:05:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fac400662f0 0x7fac400687a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac600cf1b0 0x7fac600a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac600ce780 0x7fac601eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 --2- 172.21.15.22:0/1770791406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac600cde70 0x7fac6015c890 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 >> 172.21.15.22:0/1770791406 conn(0x7fac60069960 msgr2=0x7fac600cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 shutdown_connections 2023-06-23T18:05:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:09.531+0000 7fac648c0700 1 -- 172.21.15.22:0/1770791406 wait complete. 2023-06-23T18:05:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2023-06-23T18:05:09.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-23T18:05:04.092062+0000', 'modified': '2023-06-23T18:05:07.368660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5571}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5571': {'gid': 5571, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6836/3366896912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6834', 'nonce': 3366896912}, {'type': 'v1', 'addr': '172.21.15.22:6836', 'nonce': 3366896912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:05:09.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-23T18:05:09.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-23T18:05:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 Processor -- start 2023-06-23T18:05:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 -- start start 2023-06-23T18:05:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c440cb460 0x7f0c440cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c4415c490 0x7f0c4415c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4415d430 con 0x7f0c440cb830 2023-06-23T18:05:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4415d150 con 0x7f0c440cb460 2023-06-23T18:05:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c489a0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440cd100 con 0x7f0c4415c490 2023-06-23T18:05:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c440cb460 0x7f0c440cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c43fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0c440cb830 0x7f0c440cbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34602/0 (socket says 172.21.15.22:34602) 2023-06-23T18:05:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c440cb460 0x7f0c440cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44588/0 (socket says 172.21.15.22:44588) 2023-06-23T18:05:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c43fff700 1 -- 172.21.15.22:0/602998714 learned_addr learned my addr 172.21.15.22:0/602998714 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:10.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/602998714 >> v2:172.21.15.169:3300/0 conn(0x7f0c4415c490 0x7f0c4415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 -- 172.21.15.22:0/602998714 >> v2:172.21.15.169:3300/0 conn(0x7f0c4415c490 msgr2=0x7f0c4415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 --2- 172.21.15.22:0/602998714 >> v2:172.21.15.169:3300/0 conn(0x7f0c4415c490 0x7f0c4415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 -- 172.21.15.22:0/602998714 >> v1:172.21.15.22:6789/0 conn(0x7f0c440cb830 legacy=0x7f0c440cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.075+0000 7f0c437fe700 1 -- 172.21.15.22:0/602998714 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440cd3e0 con 0x7f0c440cb460 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c437fe700 1 --2- 172.21.15.22:0/602998714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c440cb460 0x7f0c440cdbe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009d90 tx=0x7f0c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05f80a37a71edee server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c427fc700 1 -- 172.21.15.22:0/602998714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400e070 con 0x7f0c440cb460 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c427fc700 1 -- 172.21.15.22:0/602998714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400bae0 con 0x7f0c440cb460 2023-06-23T18:05:10.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c427fc700 1 -- 172.21.15.22:0/602998714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400be90 con 0x7f0c440cb460 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- 172.21.15.22:0/602998714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c440cb460 msgr2=0x7f0c440cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 --2- 172.21.15.22:0/602998714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c440cb460 0x7f0c440cdbe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009d90 tx=0x7f0c3400b330 comp rx=0 tx=0).stop 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- 172.21.15.22:0/602998714 shutdown_connections 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 --2- 172.21.15.22:0/602998714 >> v2:172.21.15.169:3300/0 conn(0x7f0c4415c490 0x7f0c4415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 --2- 172.21.15.22:0/602998714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c440cb460 0x7f0c440cdbe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- 172.21.15.22:0/602998714 >> 172.21.15.22:0/602998714 conn(0x7f0c4406aa70 msgr2=0x7f0c4406ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- 172.21.15.22:0/602998714 shutdown_connections 2023-06-23T18:05:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- 172.21.15.22:0/602998714 wait complete. 2023-06-23T18:05:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 Processor -- start 2023-06-23T18:05:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.079+0000 7f0c489a0700 1 -- start start 2023-06-23T18:05:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c440cb460 0x7f0c440c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c440cb830 0x7f0c440b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c441f1570 con 0x7f0c4415c490 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b3520 con 0x7f0c440cb830 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44155ca0 con 0x7f0c440cb460 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44594/0 (socket says 172.21.15.22:44594) 2023-06-23T18:05:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 -- 172.21.15.22:0/1759981759 learned_addr learned my addr 172.21.15.22:0/1759981759 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c437fe700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c440cb460 0x7f0c440c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c440cb830 0x7f0c440b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 -- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c440cb460 msgr2=0x7f0c440c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c440cb460 0x7f0c440c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 -- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c440cb830 msgr2=0x7f0c440b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c440cb830 0x7f0c440b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 -- 172.21.15.22:0/1759981759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34009a70 con 0x7f0c4415c490 2023-06-23T18:05:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c43fff700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c450 tx=0x7f0c3c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c2bfff700 1 -- 172.21.15.22:0/1759981759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c005d90 con 0x7f0c4415c490 2023-06-23T18:05:10.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c2bfff700 1 -- 172.21.15.22:0/1759981759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c00ce20 con 0x7f0c4415c490 2023-06-23T18:05:10.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c441f0960 con 0x7f0c4415c490 2023-06-23T18:05:10.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c2bfff700 1 -- 172.21.15.22:0/1759981759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0072d0 con 0x7f0c4415c490 2023-06-23T18:05:10.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.083+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c441f0f20 con 0x7f0c4415c490 2023-06-23T18:05:10.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c2bfff700 1 -- 172.21.15.22:0/1759981759 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f0c3c007430 con 0x7f0c4415c490 2023-06-23T18:05:10.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c2bfff700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 0x7f0c300687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c2bfff700 1 -- 172.21.15.22:0/1759981759 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0c3c082f30 con 0x7f0c4415c490 2023-06-23T18:05:10.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c437fe700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 0x7f0c300687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c437fe700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 0x7f0c300687a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f80 tx=0x7f0c34011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c4ce79740 1 Processor -- start 2023-06-23T18:05:10.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.087+0000 7f0c4ce79740 1 -- start start 2023-06-23T18:05:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 0x1bd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913710 0x1b81c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aa0690 con 0x1913eb0 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af37c0 con 0x1913ae0 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1aeca20 con 0x1913710 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44606/0 (socket says 172.21.15.22:44606) 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 -- 172.21.15.22:0/3335769817 learned_addr learned my addr 172.21.15.22:0/3335769817 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c43fff700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913710 0x1b81c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 0x1bd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 -- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913710 msgr2=0x1b81c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913710 0x1b81c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 -- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 msgr2=0x1bd0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 0x1bd0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 -- 172.21.15.22:0/3335769817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x195f9e0 con 0x1913eb0 2023-06-23T18:05:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 0x1bd0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:05:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c437fe700 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c34014040 tx=0x7f0c34010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c32abe554a8ef6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c297fa700 1 -- 172.21.15.22:0/3335769817 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34032680 con 0x1913eb0 2023-06-23T18:05:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c297fa700 1 -- 172.21.15.22:0/3335769817 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3402b070 con 0x1913eb0 2023-06-23T18:05:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c297fa700 1 -- 172.21.15.22:0/3335769817 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34032a20 con 0x1913eb0 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3335769817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 msgr2=0x1bd0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c34014040 tx=0x7f0c34010000 comp rx=0 tx=0).stop 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3335769817 shutdown_connections 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913710 0x1b81c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913ae0 0x1bd0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3335769817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd0810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3335769817 >> 172.21.15.22:0/3335769817 conn(0x1b7dab0 msgr2=0x1b880b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3335769817 shutdown_connections 2023-06-23T18:05:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3335769817 wait complete. 2023-06-23T18:05:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.091+0000 7f0c4ce79740 1 -- start start 2023-06-23T18:05:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 Processor -- start 2023-06-23T18:05:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1b8e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1b8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aaef20 con 0x1913ae0 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aad890 con 0x1913710 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1998b20 con 0x1913eb0 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 10 client.0 _ll_register_callbacks cb 0x1ab1350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1b8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44614/0 (socket says 172.21.15.22:44614) 2023-06-23T18:05:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 -- 172.21.15.22:0/2214256394 learned_addr learned my addr 172.21.15.22:0/2214256394 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c437fe700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1b8e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 -- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 msgr2=0x1b8f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1b8f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 -- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 msgr2=0x1b8e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1b8e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 -- 172.21.15.22:0/2214256394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c34022070 con 0x1913ae0 2023-06-23T18:05:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c38013560 tx=0x7f0c38013870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6a9acafa08040c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38004030 con 0x1913ae0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0c38004190 con 0x1913ae0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ae5a00 con 0x1913ae0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c38004350 con 0x1913ae0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 10 client.5766 fetch_fsmap learned FSMap version 17 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1ab2670 con 0x1913ae0 2023-06-23T18:05:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ae5a00 con 0x1913ae0 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0c380203f0 con 0x1913ae0 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 10 client.5766 fetch_fsmap finished waiting for FSMap version 17 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 10 client.5766 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 20 client.5766 populate_metadata read hostname 'smithi022' 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x195e790 0x1b86d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 4 client.5766 mds_command: new command op to 5571 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:05:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.095+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 --> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ae5720 con 0x195e790 2023-06-23T18:05:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c437fe700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x195e790 0x1b86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c437fe700 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x195e790 0x1b86d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d96656da3610c36 server_cookie=e1c84b46426e4df7 in_seq=0 out_seq=0 2023-06-23T18:05:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c227fc700 10 client.5766 ms_handle_connect on v2:172.21.15.22:6834/3366896912 2023-06-23T18:05:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c227fc700 1 -- 172.21.15.22:0/2214256394 <== mds.0 v2:172.21.15.22:6834/3366896912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x1ae5720 con 0x195e790 2023-06-23T18:05:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c227fc700 10 client.5766 handle_command_reply: tid=0 2023-06-23T18:05:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 client.5766 shutdown 2023-06-23T18:05:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 msgr2=0x1b8f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c38013560 tx=0x7f0c38013870 comp rx=0 tx=0).stop 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 shutdown_connections 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x195e790 0x1b86d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1b8f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c227fc700 0 client.5766 ms_handle_reset on v2:172.21.15.22:6834/3366896912 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1b8f030 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/2214256394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1b8e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 >> 172.21.15.22:0/2214256394 conn(0x1b7dab0 msgr2=0x1bac690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 shutdown_connections 2023-06-23T18:05:10.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 1 -- 172.21.15.22:0/2214256394 wait complete. 2023-06-23T18:05:10.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.099+0000 7f0c4ce79740 20 client.5766 trim_cache size 0 max 16384 2023-06-23T18:05:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 Processor -- start 2023-06-23T18:05:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 -- start start 2023-06-23T18:05:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 0x1bceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af0a20 con 0x1913eb0 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aef580 con 0x1913710 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ae6d20 con 0x1913ae0 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44618/0 (socket says 172.21.15.22:44618) 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c43fff700 1 -- 172.21.15.22:0/4054614995 learned_addr learned my addr 172.21.15.22:0/4054614995 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.119+0000 7f0c437fe700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 0x1bceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 -- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 msgr2=0x1bceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 0x1bceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 -- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 msgr2=0x1bce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 -- 172.21.15.22:0/4054614995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44156380 con 0x1913eb0 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 0x1bceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c43fff700 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00cab0 tx=0x7f0c3c057f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d599921b57db2b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c29ffb700 1 -- 172.21.15.22:0/4054614995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0521b0 con 0x1913eb0 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c29ffb700 1 -- 172.21.15.22:0/4054614995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c052410 con 0x1913eb0 2023-06-23T18:05:10.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c29ffb700 1 -- 172.21.15.22:0/4054614995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c05be80 con 0x1913eb0 2023-06-23T18:05:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- 172.21.15.22:0/4054614995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 msgr2=0x1bd5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00cab0 tx=0x7f0c3c057f80 comp rx=0 tx=0).stop 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- 172.21.15.22:0/4054614995 shutdown_connections 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913eb0 0x1bd5510 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913ae0 0x1bceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/4054614995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- 172.21.15.22:0/4054614995 >> 172.21.15.22:0/4054614995 conn(0x1b7dab0 msgr2=0x1b86bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- 172.21.15.22:0/4054614995 shutdown_connections 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- 172.21.15.22:0/4054614995 wait complete. 2023-06-23T18:05:10.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- start start 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 Processor -- start 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bd3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1c32090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ae8390 con 0x1913ae0 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aeb8b0 con 0x1913710 2023-06-23T18:05:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1aa6b40 con 0x1913eb0 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c4ce79740 10 client.0 _ll_register_callbacks cb 0x1a9a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.123+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bd3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1c32090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44626/0 (socket says 172.21.15.22:44626) 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bd3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.22:46912/0 (socket says 172.21.15.22:46912) 2023-06-23T18:05:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c42ffd700 1 -- 172.21.15.22:0/3498133752 learned_addr learned my addr 172.21.15.22:0/3498133752 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 -- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 msgr2=0x1bd3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bd3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 -- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 msgr2=0x1bd2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 -- 172.21.15.22:0/3498133752 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c441f0f20 con 0x1913eb0 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c42ffd700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c43fff700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1c32090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c050520 tx=0x7f0c3c00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3332ece5ddb4b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2023-06-23T18:05:10.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c067bc0 con 0x1913eb0 2023-06-23T18:05:10.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0c3c067d20 con 0x1913eb0 2023-06-23T18:05:10.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c3402b070 con 0x1913eb0 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c3c04fe60 con 0x1913eb0 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 10 client.5306 fetch_fsmap learned FSMap version 17 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1a989e0 con 0x1913eb0 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3402b070 con 0x1913eb0 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0c3c076d20 con 0x1913eb0 2023-06-23T18:05:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 10 client.5306 fetch_fsmap finished waiting for FSMap version 17 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 10 client.5306 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 20 client.5306 populate_metadata read hostname 'smithi022' 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x1bd7cd0 0x1bd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 4 client.5306 mds_command: new command op to 5571 tid=0[{"prefix": "status"}] 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 --> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x196b4e0 con 0x1bd7cd0 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c437fe700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x1bd7cd0 0x1bd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 10 client.5306 ms_handle_connect on v2:172.21.15.22:6834/3366896912 2023-06-23T18:05:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c437fe700 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x1bd7cd0 0x1bd80a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b878c0cf1f063d server_cookie=c546cf304280ceb8 in_seq=0 out_seq=0 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 1 -- 172.21.15.22:0/3498133752 <== mds.0 v2:172.21.15.22:6834/3366896912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x196b4e0 con 0x1bd7cd0 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c227fc700 10 client.5306 handle_command_reply: tid=0 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.127+0000 7f0c4ce79740 1 client.5306 shutdown 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 msgr2=0x1c32090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1c32090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c050520 tx=0x7f0c3c00cca0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 shutdown_connections 2023-06-23T18:05:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:6834/3366896912,v1:172.21.15.22:6836/3366896912] conn(0x1bd7cd0 0x1bd80a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1913eb0 0x1c32090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x1913ae0 0x1bd2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 --2- 172.21.15.22:0/3498133752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1913710 0x1bd3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c227fc700 0 client.5306 ms_handle_reset on v2:172.21.15.22:6834/3366896912 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 >> 172.21.15.22:0/3498133752 conn(0x1b7dab0 msgr2=0x1b86910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 shutdown_connections 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 1 -- 172.21.15.22:0/3498133752 wait complete. 2023-06-23T18:05:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c4ce79740 20 client.5306 trim_cache size 0 max 16384 2023-06-23T18:05:10.148 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:05:10.148 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "e4fadccd-0593-4464-bce3-4c0f7c738354", 2023-06-23T18:05:10.148 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-06-23T18:05:10.148 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5571, 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 6.0374405339999999, 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 17, 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 39, 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 39, 2023-06-23T18:05:10.149 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 13.300689252 2023-06-23T18:05:10.150 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:05:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 msgr2=0x7f0c300687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 0x7f0c300687a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f80 tx=0x7f0c34011000 comp rx=0 tx=0).stop 2023-06-23T18:05:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 msgr2=0x7f0c44155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c450 tx=0x7f0c3c00c760 comp rx=0 tx=0).stop 2023-06-23T18:05:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 shutdown_connections 2023-06-23T18:05:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c4415c490 0x7f0c44155760 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f0c300662f0 0x7f0c300687a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c440cb830 0x7f0c440b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 --2- 172.21.15.22:0/1759981759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c440cb460 0x7f0c440c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 >> 172.21.15.22:0/1759981759 conn(0x7f0c4406aa70 msgr2=0x7f0c441586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 shutdown_connections 2023-06-23T18:05:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.131+0000 7f0c489a0700 1 -- 172.21.15.22:0/1759981759 wait complete. 2023-06-23T18:05:10.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-23T18:05:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 Processor -- start 2023-06-23T18:05:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 -- start start 2023-06-23T18:05:10.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb69815beb0 0x7fb698070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 0x7fb6980c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6980ca370 con 0x7fb69806d110 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6980710e0 con 0x7fb69815beb0 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb69cf03700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980c8790 con 0x7fb69815c280 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb697fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb69806d110 0x7fb69815fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:34610/0 (socket says 172.21.15.22:34610) 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb697fff700 1 -- 172.21.15.22:0/505802712 learned_addr learned my addr 172.21.15.22:0/505802712 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 --2- 172.21.15.22:0/505802712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 0x7fb6980c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6977fe700 1 --2- 172.21.15.22:0/505802712 >> v2:172.21.15.169:3300/0 conn(0x7fb69815beb0 0x7fb698070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6967fc700 1 -- 172.21.15.22:0/505802712 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527472716 0 0) 0x7fb6980ca370 con 0x7fb69806d110 2023-06-23T18:05:10.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6967fc700 1 -- 172.21.15.22:0/505802712 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6780035d0 con 0x7fb69806d110 2023-06-23T18:05:10.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 -- 172.21.15.22:0/505802712 >> v2:172.21.15.169:3300/0 conn(0x7fb69815beb0 msgr2=0x7fb698070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:10.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 --2- 172.21.15.22:0/505802712 >> v2:172.21.15.169:3300/0 conn(0x7fb69815beb0 0x7fb698070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 -- 172.21.15.22:0/505802712 >> v1:172.21.15.22:6789/0 conn(0x7fb69806d110 legacy=0x7fb69815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 -- 172.21.15.22:0/505802712 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698160780 con 0x7fb69815c280 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb696ffd700 1 --2- 172.21.15.22:0/505802712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 0x7fb6980c8f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb698160970 tx=0x7fb680009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55ab442f679bf4b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6967fc700 1 -- 172.21.15.22:0/505802712 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680004020 con 0x7fb69815c280 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6967fc700 1 -- 172.21.15.22:0/505802712 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680004180 con 0x7fb69815c280 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.271+0000 7fb6967fc700 1 -- 172.21.15.22:0/505802712 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800044f0 con 0x7fb69815c280 2023-06-23T18:05:10.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- 172.21.15.22:0/505802712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 msgr2=0x7fb6980c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- 172.21.15.22:0/505802712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 0x7fb6980c8f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb698160970 tx=0x7fb680009b30 comp rx=0 tx=0).stop 2023-06-23T18:05:10.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- 172.21.15.22:0/505802712 shutdown_connections 2023-06-23T18:05:10.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- 172.21.15.22:0/505802712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815c280 0x7fb6980c8f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- 172.21.15.22:0/505802712 >> v2:172.21.15.169:3300/0 conn(0x7fb69815beb0 0x7fb698070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- 172.21.15.22:0/505802712 >> 172.21.15.22:0/505802712 conn(0x7fb69806a5f0 msgr2=0x7fb6980caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- 172.21.15.22:0/505802712 shutdown_connections 2023-06-23T18:05:10.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- 172.21.15.22:0/505802712 wait complete. 2023-06-23T18:05:10.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 Processor -- start 2023-06-23T18:05:10.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- start start 2023-06-23T18:05:10.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815beb0 0x7fb6981f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb69815c280 0x7fb69815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980b3250 con 0x7fb69806d110 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69815e7c0 con 0x7fb69815c280 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb69cf03700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69815ddf0 con 0x7fb69815beb0 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44630/0 (socket says 172.21.15.22:44630) 2023-06-23T18:05:10.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 -- 172.21.15.22:0/1410190143 learned_addr learned my addr 172.21.15.22:0/1410190143 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb696ffd700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815beb0 0x7fb6981f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb697fff700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb69815c280 0x7fb69815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 -- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815beb0 msgr2=0x7fb6981f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815beb0 0x7fb6981f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 -- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb69815c280 msgr2=0x7fb69815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb69815c280 0x7fb69815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.275+0000 7fb6977fe700 1 -- 172.21.15.22:0/1410190143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800094b0 con 0x7fb69806d110 2023-06-23T18:05:10.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb6977fe700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb690009d10 tx=0x7fb69000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69000cc90 con 0x7fb69806d110 2023-06-23T18:05:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69815d1e0 con 0x7fb69806d110 2023-06-23T18:05:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69000f040 con 0x7fb69806d110 2023-06-23T18:05:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690009e60 con 0x7fb69806d110 2023-06-23T18:05:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69815d7a0 con 0x7fb69806d110 2023-06-23T18:05:10.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6980c4f30 con 0x7fb69806d110 2023-06-23T18:05:10.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.279+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fb690014480 con 0x7fb69806d110 2023-06-23T18:05:10.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.283+0000 7fb68ffff700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 0x7fb67c0687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:10.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.283+0000 7fb696ffd700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 0x7fb67c0687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:10.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.283+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb690084550 con 0x7fb69806d110 2023-06-23T18:05:10.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.283+0000 7fb696ffd700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 0x7fb67c0687a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb6800090b0 tx=0x7fb680008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:10.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.287+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7fb6900518a0 con 0x7fb69806d110 2023-06-23T18:05:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb69804e4c0 con 0x7fb69806d110 2023-06-23T18:05:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb68ffff700 1 -- 172.21.15.22:0/1410190143 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb690054ec0 con 0x7fb69806d110 2023-06-23T18:05:10.506 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:05:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 msgr2=0x7fb67c0687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 0x7fb67c0687a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb6800090b0 tx=0x7fb680008030 comp rx=0 tx=0).stop 2023-06-23T18:05:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 msgr2=0x7fb69815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb690009d10 tx=0x7fb69000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 shutdown_connections 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7fb67c0662f0 0x7fb67c0687a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.495+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb69815c280 0x7fb69815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.499+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb69815beb0 0x7fb6981f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.499+0000 7fb69cf03700 1 --2- 172.21.15.22:0/1410190143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb69806d110 0x7fb69815e280 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.499+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 >> 172.21.15.22:0/1410190143 conn(0x7fb69806a5f0 msgr2=0x7fb6980acdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.499+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 shutdown_connections 2023-06-23T18:05:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:10.499+0000 7fb69cf03700 1 -- 172.21.15.22:0/1410190143 wait complete. 2023-06-23T18:05:10.521 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:05:10.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:05:10.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-06-23T18:05:10.522 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-23T18:05:10.522 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:10.522 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:05:10.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:05:10.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:05:10.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:05:10.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.524 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-06-23T18:05:10.530 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:05:10.530 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:05:10.530 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:05:10.530 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.530 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:05:10.531 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.532 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-06-23T18:05:10.532 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42494sec preferred_lft 42494sec 2023-06-23T18:05:10.532 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-06-23T18:05:10.532 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:c1:01:7a:e8 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-23T18:05:10.533 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 2e:51:f1:51:a4:ce brd ff:ff:ff:ff:ff:ff 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::a807:d9ff:fe1b:169a/64 scope link 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 2e:51:f1:51:a4:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-23T18:05:10.534 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::2c51:f1ff:fe51:a4ce/64 scope link 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 92:0b:e5:39:3d:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::900b:e5ff:fe39:3de9/64 scope link 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:05:10.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.536 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:05:10.550 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-06-23T18:05:10.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.552 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-06-23T18:05:10.558 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-06-23T18:05:10.558 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:05:10.559 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-06-23T18:05:10.559 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-06-23T18:05:10.559 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-06-23T18:05:10.559 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:05:10.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.560 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-06-23T18:05:10.560 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-06-23T18:05:10.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-23T18:05:10.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-23T18:05:10.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-23T18:05:10.561 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-06-23T18:05:10.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.645 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-06-23T18:05:10.651 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-23T18:05:10.651 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-23T18:05:10.651 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:05:10.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-06-23T18:05:10.666 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-23T18:05:10.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:05:10.666 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:10.715 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-23T18:05:10.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:05:10.715 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:10.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:05:10.740 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:05:10.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:05:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:05:10.800 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:05:10.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.801 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:05:10.808 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:05:10.809 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-23T18:05:10.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:05:10.831 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:05:10.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:05:10.847 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:05:10.848 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:05:10.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:10.850 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:05:10.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2023-06-23T18:05:10.887+0000 7f1c336ae640 -1 init, newargv = 0x5650226b8b90 newargc=15 2023-06-23T18:05:10.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[47062]: starting ceph client 2023-06-23T18:05:10.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[47062]: starting fuse 2023-06-23T18:05:12.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-06-23T18:05:12.022 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:05:12.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:05:12.079 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:05:12.080 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:05:12.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.082 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-06-23T18:05:12.091 INFO:teuthology.orchestra.run.smithi022.stdout:52 2023-06-23T18:05:12.091 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-06-23T18:05:12.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:05:12.093 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-06-23T18:05:12.093 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-06-23T18:05:12.093 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-06-23T18:05:12.094 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-06-23T18:05:12.094 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:05:12.095 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-06-23T18:05:12.096 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-23T18:05:12.096 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-06-23T18:05:12.097 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-23T18:05:12.097 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:05:12.098 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-06-23T18:05:12.098 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-06-23T18:05:12.098 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-06-23T18:05:12.099 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-06-23T18:05:12.099 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:05:12.099 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-06-23T18:05:12.100 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:05:12.100 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:05:12.100 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:05:12.101 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-06-23T18:05:12.101 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-06-23T18:05:12.101 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-06-23T18:05:12.102 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:05:12.102 DEBUG:teuthology.orchestra.run.smithi022:> 2023-06-23T18:05:12.102 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2023-06-23T18:05:12.103 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-06-23T18:05:12.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.47062.asok 2023-06-23T18:05:12.374 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47062.asok 2023-06-23T18:05:12.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47062.asok status 2023-06-23T18:05:12.523 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-06-23T18:05:12.523 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-06-23T18:05:12.523 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:05:12.524 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:05:12.524 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2023-06-23T18:05:12.524 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-06-23T18:05:12.524 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-23T18:05:12.524 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "47062", 2023-06-23T18:05:12.525 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-06-23T18:05:12.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:05:12.525 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5784, 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-06-23T18:05:12.526 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 5784 2023-06-23T18:05:12.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:05:12.527 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:05:12.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:05:12.527 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:05:12.528 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3777408255 2023-06-23T18:05:12.528 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-06-23T18:05:12.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:05:12.528 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-06-23T18:05:12.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-06-23T18:05:12.529 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:05:12.529 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3777408255 2023-06-23T18:05:12.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-06-23T18:05:12.530 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.5784 192.168.0.1:0/3777408255", 2023-06-23T18:05:12.530 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/3777408255", 2023-06-23T18:05:12.530 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-06-23T18:05:12.531 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 17, 2023-06-23T18:05:12.531 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 39, 2023-06-23T18:05:12.531 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:05:12.531 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-06-23T18:05:12.532 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-06-23T18:05:12.532 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-06-23T18:05:12.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.535 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-06-23T18:05:12.535 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:12.542 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-06-23T18:05:12.543 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:12.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-23T18:05:12.563 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=xfstests-dev 2023-06-23T18:05:12.610 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.420M2hxqAdxfstests-dev 2023-06-23T18:05:12.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.611 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.420M2hxqAdxfstests-dev) 2023-06-23T18:05:12.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:12.633 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.420M2hxqAdxfstests-dev) 2023-06-23T18:05:12.650 INFO:teuthology.orchestra.run.smithi022.stderr:Cloning into '/tmp/tmp.420M2hxqAdxfstests-dev'... 2023-06-23T18:05:13.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:13.369 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-06-23T18:05:13.431 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=test 2023-06-23T18:05:13.438 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.qA86lK7wK3test 2023-06-23T18:05:13.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:05:13.439 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-06-23T18:05:13.452 DEBUG:teuthology.orchestra.run.smithi022:> mktemp -d --suffix=scratch 2023-06-23T18:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.I2APt55VJnscratch 2023-06-23T18:05:13.502 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "20.04". 2023-06-23T18:05:13.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get install -y xfslibs-dev uuid-dev libtool-bin e2fsprogs automake gcc libuuid1 quota attr libattr1-dev make libacl1-dev libaio-dev xfsprogs libgdbm-dev gawk fio dbench uuid-runtime python2 sqlite3 2023-06-23T18:05:13.578 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2023-06-23T18:05:13.784 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2023-06-23T18:05:13.785 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2023-06-23T18:05:13.971 INFO:teuthology.orchestra.run.smithi022.stdout:attr is already the newest version (1:2.4.48-5). 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:automake is already the newest version (1:1.16.1-4ubuntu6). 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:gawk is already the newest version (1:5.0.1+dfsg-1). 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:gcc is already the newest version (4:9.3.0-1ubuntu2). 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:gcc set to manually installed. 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:libacl1-dev is already the newest version (2.2.53-6). 2023-06-23T18:05:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:libaio-dev is already the newest version (0.3.112-5). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:libattr1-dev is already the newest version (1:2.4.48-5). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:make is already the newest version (4.2.1-1.2). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:make set to manually installed. 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:quota is already the newest version (4.05-1). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:xfsprogs is already the newest version (5.3.0-1ubuntu2). 2023-06-23T18:05:13.973 INFO:teuthology.orchestra.run.smithi022.stdout:dbench is already the newest version (4.0-2build1). 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:python2 is already the newest version (2.7.17-2ubuntu4). 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:python2 set to manually installed. 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:e2fsprogs is already the newest version (1.45.5-2ubuntu1.1). 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:libuuid1 is already the newest version (2.34-0.1ubuntu9.4). 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.4). 2023-06-23T18:05:13.974 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2023-06-23T18:05:13.975 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-06-23T18:05:13.975 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2023-06-23T18:05:13.975 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2023-06-23T18:05:13.975 INFO:teuthology.orchestra.run.smithi022.stdout: libgfapi0 libgfrpc0 libgfxdr0 libglusterfs0 libsqlite3-0 2023-06-23T18:05:13.976 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2023-06-23T18:05:13.976 INFO:teuthology.orchestra.run.smithi022.stdout: gnuplot gfio python-scipy sqlite3-doc 2023-06-23T18:05:14.008 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2023-06-23T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stdout: fio libgdbm-dev libgfapi0 libgfrpc0 libgfxdr0 libglusterfs0 sqlite3 2023-06-23T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2023-06-23T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stdout: libsqlite3-0 uuid-runtime 2023-06-23T18:05:14.190 INFO:teuthology.orchestra.run.smithi022.stdout:2 upgraded, 7 newly installed, 0 to remove and 199 not upgraded. 2023-06-23T18:05:14.190 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 2,464 kB of archives. 2023-06-23T18:05:14.191 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 7,043 kB of additional disk space will be used. 2023-06-23T18:05:14.191 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-runtime amd64 2.34-0.1ubuntu9.4 [35.4 kB] 2023-06-23T18:05:14.392 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsqlite3-0 amd64 3.31.1-4ubuntu0.5 [549 kB] 2023-06-23T18:05:14.737 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 libglusterfs0 amd64 7.2-2build1 [286 kB] 2023-06-23T18:05:14.788 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgfxdr0 amd64 7.2-2build1 [24.2 kB] 2023-06-23T18:05:14.791 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgfrpc0 amd64 7.2-2build1 [46.1 kB] 2023-06-23T18:05:14.797 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libgfapi0 amd64 7.2-2build1 [72.6 kB] 2023-06-23T18:05:14.806 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 fio amd64 3.16-1 [507 kB] 2023-06-23T18:05:14.853 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libgdbm-dev amd64 1.18.1-5 [83.4 kB] 2023-06-23T18:05:14.859 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 sqlite3 amd64 3.31.1-4ubuntu0.5 [860 kB] 2023-06-23T18:05:15.187 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: unable to initialize frontend: Dialog 2023-06-23T18:05:15.188 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2023-06-23T18:05:15.188 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: falling back to frontend: Readline 2023-06-23T18:05:15.204 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: unable to initialize frontend: Readline 2023-06-23T18:05:15.204 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: (This frontend requires a controlling tty.) 2023-06-23T18:05:15.204 INFO:teuthology.orchestra.run.smithi022.stderr:debconf: falling back to frontend: Teletype 2023-06-23T18:05:15.211 INFO:teuthology.orchestra.run.smithi022.stderr:dpkg-preconfigure: unable to re-open stdin: 2023-06-23T18:05:15.270 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 2,464 kB in 1s (2,759 kB/s) 2023-06-23T18:05:15.551 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 155316 files and directories currently installed.) 2023-06-23T18:05:15.552 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-uuid-runtime_2.34-0.1ubuntu9.4_amd64.deb ... 2023-06-23T18:05:15.683 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking uuid-runtime (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-06-23T18:05:16.165 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-libsqlite3-0_3.31.1-4ubuntu0.5_amd64.deb ... 2023-06-23T18:05:16.342 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libsqlite3-0:amd64 (3.31.1-4ubuntu0.5) over (3.31.1-4ubuntu0.3) ... 2023-06-23T18:05:16.746 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libglusterfs0:amd64. 2023-06-23T18:05:16.765 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-libglusterfs0_7.2-2build1_amd64.deb ... 2023-06-23T18:05:16.829 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libglusterfs0:amd64 (7.2-2build1) ... 2023-06-23T18:05:17.175 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libgfxdr0:amd64. 2023-06-23T18:05:17.193 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-libgfxdr0_7.2-2build1_amd64.deb ... 2023-06-23T18:05:17.249 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libgfxdr0:amd64 (7.2-2build1) ... 2023-06-23T18:05:17.553 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libgfrpc0:amd64. 2023-06-23T18:05:17.575 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-libgfrpc0_7.2-2build1_amd64.deb ... 2023-06-23T18:05:17.611 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libgfrpc0:amd64 (7.2-2build1) ... 2023-06-23T18:05:17.915 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libgfapi0:amd64. 2023-06-23T18:05:17.935 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-libgfapi0_7.2-2build1_amd64.deb ... 2023-06-23T18:05:17.972 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libgfapi0:amd64 (7.2-2build1) ... 2023-06-23T18:05:18.318 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package fio. 2023-06-23T18:05:18.338 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-fio_3.16-1_amd64.deb ... 2023-06-23T18:05:18.401 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking fio (3.16-1) ... 2023-06-23T18:05:18.739 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2023-06-23T18:05:18.760 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-libgdbm-dev_1.18.1-5_amd64.deb ... 2023-06-23T18:05:18.796 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libgdbm-dev:amd64 (1.18.1-5) ... 2023-06-23T18:05:19.167 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package sqlite3. 2023-06-23T18:05:19.186 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../8-sqlite3_3.31.1-4ubuntu0.5_amd64.deb ... 2023-06-23T18:05:19.225 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking sqlite3 (3.31.1-4ubuntu0.5) ... 2023-06-23T18:05:19.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:05:19.289 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:05:19.612 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libsqlite3-0:amd64 (3.31.1-4ubuntu0.5) ... 2023-06-23T18:05:19.738 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libglusterfs0:amd64 (7.2-2build1) ... 2023-06-23T18:05:19.856 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up uuid-runtime (2.34-0.1ubuntu9.4) ... 2023-06-23T18:05:20.784 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libgdbm-dev:amd64 (1.18.1-5) ... 2023-06-23T18:05:20.918 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up sqlite3 (3.31.1-4ubuntu0.5) ... 2023-06-23T18:05:21.044 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libgfxdr0:amd64 (7.2-2build1) ... 2023-06-23T18:05:21.171 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libgfrpc0:amd64 (7.2-2build1) ... 2023-06-23T18:05:21.297 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libgfapi0:amd64 (7.2-2build1) ... 2023-06-23T18:05:21.423 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up fio (3.16-1) ... 2023-06-23T18:05:21.701 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-06-23T18:05:21.852 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-06-23T18:05:22.154 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-06-23T18:05:23.997 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-06-23T18:05:24.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd -m fsgqa 2023-06-23T18:05:24.644 INFO:teuthology.orchestra.run.smithi022.stderr:useradd: user 'fsgqa' already exists 2023-06-23T18:05:24.645 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-23T18:05:24.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo groupadd fsgqa 2023-06-23T18:05:24.701 INFO:teuthology.orchestra.run.smithi022.stderr:groupadd: group 'fsgqa' already exists 2023-06-23T18:05:24.702 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-23T18:05:24.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd fsgqa2 2023-06-23T18:05:25.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo useradd 123456-fsgqa 2023-06-23T18:05:25.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-06-23T18:05:25.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 Processor -- start 2023-06-23T18:05:25.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- start start 2023-06-23T18:05:25.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f96400c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9640160420 0x7f96400cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400c18f0 con 0x7f9640070040 2023-06-23T18:05:25.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9640163ec0 con 0x7f9640160420 2023-06-23T18:05:25.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96401641a0 con 0x7f96400c0fe0 2023-06-23T18:05:25.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9640160420 0x7f96400cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9640160420 0x7f96400cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54476/0 (socket says 172.21.15.22:54476) 2023-06-23T18:05:25.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 -- 172.21.15.22:0/2942719439 learned_addr learned my addr 172.21.15.22:0/2942719439 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:25.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9645e77700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9640070040 0x7f96400afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:55180/0 (socket says 172.21.15.22:55180) 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9645676700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f96400c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 -- 172.21.15.22:0/2942719439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 msgr2=0x7f96400c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f96400c13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 -- 172.21.15.22:0/2942719439 >> v1:172.21.15.22:6789/0 conn(0x7f9640070040 legacy=0x7f96400afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 -- 172.21.15.22:0/2942719439 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640164480 con 0x7f9640160420 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9644e75700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640160420 0x7f96400cbc70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9630009e50 tx=0x7f963000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a83c86b19cb354 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f963ffff700 1 -- 172.21.15.22:0/2942719439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000e070 con 0x7f9640160420 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f963ffff700 1 -- 172.21.15.22:0/2942719439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963000b910 con 0x7f9640160420 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f963ffff700 1 -- 172.21.15.22:0/2942719439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000bcc0 con 0x7f9640160420 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- 172.21.15.22:0/2942719439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640160420 msgr2=0x7f96400cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640160420 0x7f96400cbc70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9630009e50 tx=0x7f963000b330 comp rx=0 tx=0).stop 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- 172.21.15.22:0/2942719439 shutdown_connections 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640160420 0x7f96400cbc70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 --2- 172.21.15.22:0/2942719439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f96400c13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.547+0000 7f9646678700 1 -- 172.21.15.22:0/2942719439 >> 172.21.15.22:0/2942719439 conn(0x7f964006a5f0 msgr2=0x7f96400caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:25.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- 172.21.15.22:0/2942719439 shutdown_connections 2023-06-23T18:05:25.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- 172.21.15.22:0/2942719439 wait complete. 2023-06-23T18:05:25.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 Processor -- start 2023-06-23T18:05:25.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- start start 2023-06-23T18:05:25.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f964015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9640160420 0x7f96401f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9640098e20 con 0x7f9640070040 2023-06-23T18:05:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96401f69a0 con 0x7f9640160420 2023-06-23T18:05:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9640161e50 con 0x7f96400c0fe0 2023-06-23T18:05:25.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54488/0 (socket says 172.21.15.22:54488) 2023-06-23T18:05:25.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 -- 172.21.15.22:0/551276066 learned_addr learned my addr 172.21.15.22:0/551276066 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:25.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9644e75700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f964015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645e77700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9640160420 0x7f96401f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 -- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 msgr2=0x7f964015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f964015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 -- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9640160420 msgr2=0x7f96401f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9640160420 0x7f96401f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 -- 172.21.15.22:0/551276066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009b30 con 0x7f9640070040 2023-06-23T18:05:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9645676700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f962800e9d0 tx=0x7f962800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628009720 con 0x7f9640070040 2023-06-23T18:05:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800ee70 con 0x7f9640070040 2023-06-23T18:05:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9640162130 con 0x7f9640070040 2023-06-23T18:05:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800ca50 con 0x7f9640070040 2023-06-23T18:05:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.551+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401626f0 con 0x7f9640070040 2023-06-23T18:05:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964004e4c0 con 0x7f9640070040 2023-06-23T18:05:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f962800cbb0 con 0x7f9640070040 2023-06-23T18:05:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f963dffb700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 0x7f96200687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9628083ec0 con 0x7f9640070040 2023-06-23T18:05:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f9644e75700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 0x7f96200687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.555+0000 7f9644e75700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 0x7f96200687a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f963000b370 tx=0x7f9630009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.567+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f9628054c90 con 0x7f9640070040 2023-06-23T18:05:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.791+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f96400b1630 con 0x7f9640070040 2023-06-23T18:05:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f963dffb700 1 -- 172.21.15.22:0/551276066 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f9628021510 con 0x7f9640070040 2023-06-23T18:05:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:05:25.804 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"fsid":"e4fadccd-0593-4464-bce3-4c0f7c738354","modified":"2023-06-23T18:02:49.475158Z","created":"2023-06-23T18:02:49.475158Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3301","nonce":0},{"type":"v1","addr":"172.21.15.169:6790","nonce":0}]},"addr":"172.21.15.169:6790/0","public_addr":"172.21.15.169:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-06-23T18:05:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 msgr2=0x7f96200687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 0x7f96200687a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f963000b370 tx=0x7f9630009000 comp rx=0 tx=0).stop 2023-06-23T18:05:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 msgr2=0x7f9640161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f962800e9d0 tx=0x7f962800ece0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 shutdown_connections 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f96200662f0 0x7f96200687a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9640160420 0x7f96401f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96400c0fe0 0x7f964015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 --2- 172.21.15.22:0/551276066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9640070040 0x7f9640161870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 >> 172.21.15.22:0/551276066 conn(0x7f964006a5f0 msgr2=0x7f96400c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 shutdown_connections 2023-06-23T18:05:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.795+0000 7f9646678700 1 -- 172.21.15.22:0/551276066 wait complete. 2023-06-23T18:05:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:dumped monmap epoch 1 2023-06-23T18:05:25.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2023-06-23T18:05:25.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.923+0000 7f75a8a00700 1 Processor -- start 2023-06-23T18:05:25.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.923+0000 7f75a8a00700 1 -- start start 2023-06-23T18:05:25.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f75a40cb460 0x7f75a40cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75a415c490 0x7f75a415c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a415d430 con 0x7f75a40cb460 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a415d150 con 0x7f75a415c490 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40cd100 con 0x7f75a40cb830 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f75a40cb460 0x7f75a40cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f75a40cb460 0x7f75a40cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54500/0 (socket says 172.21.15.22:54500) 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2838098742 learned_addr learned my addr 172.21.15.22:0/2838098742 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:25.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a359e700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a259c700 1 --2- 172.21.15.22:0/2838098742 >> v2:172.21.15.169:3300/0 conn(0x7f75a415c490 0x7f75a415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2838098742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 msgr2=0x7f75a40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:25.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2838098742 >> v2:172.21.15.169:3300/0 conn(0x7f75a415c490 msgr2=0x7f75a415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2838098742 >> v2:172.21.15.169:3300/0 conn(0x7f75a415c490 0x7f75a415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2838098742 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40cd3e0 con 0x7f75a40cb460 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a359e700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40cdbe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f759c009e50 tx=0x7f759c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba32b2ddc4321a5b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a1d9b700 1 -- 172.21.15.22:0/2838098742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00e070 con 0x7f75a40cb460 2023-06-23T18:05:25.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a1d9b700 1 -- 172.21.15.22:0/2838098742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c00b980 con 0x7f75a40cb460 2023-06-23T18:05:25.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a1d9b700 1 -- 172.21.15.22:0/2838098742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00bd30 con 0x7f75a40cb460 2023-06-23T18:05:25.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- 172.21.15.22:0/2838098742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 msgr2=0x7f75a40cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40cdbe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f759c009e50 tx=0x7f759c00b330 comp rx=0 tx=0).stop 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- 172.21.15.22:0/2838098742 shutdown_connections 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2838098742 >> v2:172.21.15.169:3300/0 conn(0x7f75a415c490 0x7f75a415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2838098742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40cdbe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- 172.21.15.22:0/2838098742 >> 172.21.15.22:0/2838098742 conn(0x7f75a406aa70 msgr2=0x7f75a406ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- 172.21.15.22:0/2838098742 shutdown_connections 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.927+0000 7f75a8a00700 1 -- 172.21.15.22:0/2838098742 wait complete. 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 Processor -- start 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 -- start start 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a4144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a415c490 0x7f75a41451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b4e30 con 0x7f75a40cb460 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4151c30 con 0x7f75a415c490 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a8a00700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b2030 con 0x7f75a40cb830 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54508/0 (socket says 172.21.15.22:54508) 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2846244555 learned_addr learned my addr 172.21.15.22:0/2846244555 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a259c700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a4144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a359e700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a415c490 0x7f75a41451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 msgr2=0x7f75a4144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a4144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a415c490 msgr2=0x7f75a41451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a415c490 0x7f75a41451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.931+0000 7f75a2d9d700 1 -- 172.21.15.22:0/2846244555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009b30 con 0x7f75a40cb460 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f75a2d9d700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b370 tx=0x7f759c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0099b0 con 0x7f75a40cb460 2023-06-23T18:05:25.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c004250 con 0x7f75a40cb460 2023-06-23T18:05:25.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a40b2310 con 0x7f75a40cb460 2023-06-23T18:05:25.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0045e0 con 0x7f75a40cb460 2023-06-23T18:05:25.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a40b28d0 con 0x7f75a40cb460 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a40cc1b0 con 0x7f75a40cb460 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.935+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7f759c004740 con 0x7f75a40cb460 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.939+0000 7f759b7fe700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 0x7f75840687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.939+0000 7f75a259c700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 0x7f75840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.939+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f759c085610 con 0x7f75a40cb460 2023-06-23T18:05:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.939+0000 7f75a259c700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 0x7f75840687a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75a414eb90 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:05:25.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:25.947+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169708 (secure 0 0 0) 0x7f759c052940 con 0x7f75a40cb460 2023-06-23T18:05:26.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.167+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) v1 -- 0x7f75a404e4c0 con 0x7f75a40cb460 2023-06-23T18:05:26.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.167+0000 7f759b7fe700 1 -- 172.21.15.22:0/2846244555 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v13) v1 ==== 82+0+151 (secure 0 0 0) 0x7f759c055f60 con 0x7f75a40cb460 2023-06-23T18:05:26.178 INFO:teuthology.orchestra.run.smithi022.stdout:[client.admin] 2023-06-23T18:05:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBJ3pVkBHqWJBAA/ld6gd35pc/K63GqX4+3jQ== 2023-06-23T18:05:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow *" 2023-06-23T18:05:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: caps mgr = "allow *" 2023-06-23T18:05:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow *" 2023-06-23T18:05:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow *" 2023-06-23T18:05:26.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 msgr2=0x7f75840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:26.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 0x7f75840687a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75a414eb90 tx=0x7f758c008040 comp rx=0 tx=0).stop 2023-06-23T18:05:26.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 msgr2=0x7f75a40b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b370 tx=0x7f759c004af0 comp rx=0 tx=0).stop 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 shutdown_connections 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:6832/3032843375,v1:172.21.15.22:6833/3032843375] conn(0x7f75840662f0 0x7f75840687a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a415c490 0x7f75a41451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75a40cb830 0x7f75a4144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 --2- 172.21.15.22:0/2846244555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a40cb460 0x7f75a40b48f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:05:26.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 >> 172.21.15.22:0/2846244555 conn(0x7f75a406aa70 msgr2=0x7f75a40ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:05:26.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 shutdown_connections 2023-06-23T18:05:26.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-06-23T18:05:26.171+0000 7f75a8a00700 1 -- 172.21.15.22:0/2846244555 wait complete. 2023-06-23T18:05:26.193 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2023-06-23T18:05:26.199 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.kYe8dspnS6 2023-06-23T18:05:26.199 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:05:26.200 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/tmp/tmp.kYe8dspnS6 2023-06-23T18:05:26.206 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:05:26.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/tmp/tmp.420M2hxqAdxfstests-dev/local.config 2023-06-23T18:05:26.220 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.qA86lK7wK3test export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.I2APt55VJnscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.22:6789 -k /tmp/tmp.kYe8dspnS6 --client_mountpoint /test" 2023-06-23T18:05:26.220 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:05:26.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/tmp/tmp.420M2hxqAdxfstests-dev/ceph.exclude 2023-06-23T18:05:26.231 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo make) 2023-06-23T18:05:26.258 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize -cfi 2023-06-23T18:05:26.742 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: putting auxiliary files in '.'. 2023-06-23T18:05:26.743 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './config.guess' 2023-06-23T18:05:26.753 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './config.sub' 2023-06-23T18:05:26.770 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './install-sh' 2023-06-23T18:05:26.782 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file './ltmain.sh' 2023-06-23T18:05:26.828 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-06-23T18:05:26.828 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-06-23T18:05:26.844 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-06-23T18:05:26.855 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-06-23T18:05:26.866 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-06-23T18:05:26.877 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-06-23T18:05:26.888 INFO:teuthology.orchestra.run.smithi022.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-06-23T18:05:26.889 INFO:teuthology.orchestra.run.smithi022.stdout:cp include/install-sh . 2023-06-23T18:05:26.891 INFO:teuthology.orchestra.run.smithi022.stdout:aclocal -I m4 2023-06-23T18:05:28.370 INFO:teuthology.orchestra.run.smithi022.stdout:autoheader 2023-06-23T18:05:28.728 INFO:teuthology.orchestra.run.smithi022.stdout:autoconf 2023-06-23T18:05:28.959 INFO:teuthology.orchestra.run.smithi022.stdout:./configure \ 2023-06-23T18:05:28.959 INFO:teuthology.orchestra.run.smithi022.stdout: --libexecdir=/usr/lib \ 2023-06-23T18:05:28.960 INFO:teuthology.orchestra.run.smithi022.stdout: --exec_prefix=/var/lib 2023-06-23T18:05:29.664 INFO:teuthology.orchestra.run.smithi022.stdout:checking build system type... x86_64-pc-linux-gnu 2023-06-23T18:05:29.664 INFO:teuthology.orchestra.run.smithi022.stdout:checking host system type... x86_64-pc-linux-gnu 2023-06-23T18:05:29.707 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to print strings... printf 2023-06-23T18:05:29.709 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc... gcc 2023-06-23T18:05:29.932 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the C compiler works... yes 2023-06-23T18:05:29.933 INFO:teuthology.orchestra.run.smithi022.stdout:checking for C compiler default output file name... a.out 2023-06-23T18:05:29.987 INFO:teuthology.orchestra.run.smithi022.stdout:checking for suffix of executables... 2023-06-23T18:05:30.090 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether we are cross compiling... no 2023-06-23T18:05:30.120 INFO:teuthology.orchestra.run.smithi022.stdout:checking for suffix of object files... o 2023-06-23T18:05:30.151 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether we are using the GNU C compiler... yes 2023-06-23T18:05:30.184 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc accepts -g... yes 2023-06-23T18:05:30.267 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to accept ISO C89... none needed 2023-06-23T18:05:30.273 INFO:teuthology.orchestra.run.smithi022.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2023-06-23T18:05:30.278 INFO:teuthology.orchestra.run.smithi022.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-06-23T18:05:30.281 INFO:teuthology.orchestra.run.smithi022.stdout:checking for egrep... /usr/bin/grep -E 2023-06-23T18:05:30.284 INFO:teuthology.orchestra.run.smithi022.stdout:checking for fgrep... /usr/bin/grep -F 2023-06-23T18:05:30.290 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ld used by gcc... /usr/bin/ld 2023-06-23T18:05:30.295 INFO:teuthology.orchestra.run.smithi022.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2023-06-23T18:05:30.306 INFO:teuthology.orchestra.run.smithi022.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2023-06-23T18:05:30.344 INFO:teuthology.orchestra.run.smithi022.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2023-06-23T18:05:30.344 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether ln -s works... yes 2023-06-23T18:05:30.360 INFO:teuthology.orchestra.run.smithi022.stdout:checking the maximum length of command line arguments... 1572864 2023-06-23T18:05:30.364 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-06-23T18:05:30.365 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-06-23T18:05:30.366 INFO:teuthology.orchestra.run.smithi022.stdout:checking for /usr/bin/ld option to reload object files... -r 2023-06-23T18:05:30.367 INFO:teuthology.orchestra.run.smithi022.stdout:checking for objdump... objdump 2023-06-23T18:05:30.368 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to recognize dependent libraries... pass_all 2023-06-23T18:05:30.370 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dlltool... no 2023-06-23T18:05:30.371 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-06-23T18:05:30.372 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ar... ar 2023-06-23T18:05:30.414 INFO:teuthology.orchestra.run.smithi022.stdout:checking for archiver @FILE support... @ 2023-06-23T18:05:30.415 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strip... strip 2023-06-23T18:05:30.416 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ranlib... ranlib 2023-06-23T18:05:30.416 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gawk... gawk 2023-06-23T18:05:30.531 INFO:teuthology.orchestra.run.smithi022.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2023-06-23T18:05:30.534 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sysroot... no 2023-06-23T18:05:30.541 INFO:teuthology.orchestra.run.smithi022.stdout:checking for a working dd... /usr/bin/dd 2023-06-23T18:05:30.547 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2023-06-23T18:05:30.572 INFO:teuthology.orchestra.run.smithi022.stdout:checking for mt... mt 2023-06-23T18:05:30.592 INFO:teuthology.orchestra.run.smithi022.stdout:checking if mt is a manifest tool... no 2023-06-23T18:05:30.656 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to run the C preprocessor... gcc -E 2023-06-23T18:05:30.865 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ANSI C header files... yes 2023-06-23T18:05:30.908 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/types.h... yes 2023-06-23T18:05:30.959 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/stat.h... yes 2023-06-23T18:05:31.005 INFO:teuthology.orchestra.run.smithi022.stdout:checking for stdlib.h... yes 2023-06-23T18:05:31.056 INFO:teuthology.orchestra.run.smithi022.stdout:checking for string.h... yes 2023-06-23T18:05:31.109 INFO:teuthology.orchestra.run.smithi022.stdout:checking for memory.h... yes 2023-06-23T18:05:31.156 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strings.h... yes 2023-06-23T18:05:31.214 INFO:teuthology.orchestra.run.smithi022.stdout:checking for inttypes.h... yes 2023-06-23T18:05:31.268 INFO:teuthology.orchestra.run.smithi022.stdout:checking for stdint.h... yes 2023-06-23T18:05:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:checking for unistd.h... yes 2023-06-23T18:05:31.402 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dlfcn.h... yes 2023-06-23T18:05:31.425 INFO:teuthology.orchestra.run.smithi022.stdout:checking for objdir... .libs 2023-06-23T18:05:31.544 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-06-23T18:05:31.545 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-06-23T18:05:31.577 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-06-23T18:05:31.740 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc static flag -static works... yes 2023-06-23T18:05:31.785 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -c -o file.o... yes 2023-06-23T18:05:31.785 INFO:teuthology.orchestra.run.smithi022.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-06-23T18:05:31.825 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-06-23T18:05:31.868 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether -lc should be explicitly linked in... no 2023-06-23T18:05:31.958 INFO:teuthology.orchestra.run.smithi022.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-06-23T18:05:31.958 INFO:teuthology.orchestra.run.smithi022.stdout:checking how to hardcode library paths into programs... immediate 2023-06-23T18:05:31.981 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether stripping libraries is possible... yes 2023-06-23T18:05:31.982 INFO:teuthology.orchestra.run.smithi022.stdout:checking if libtool supports shared libraries... yes 2023-06-23T18:05:31.982 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether to build shared libraries... yes 2023-06-23T18:05:31.983 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether to build static libraries... yes 2023-06-23T18:05:32.001 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc... (cached) gcc 2023-06-23T18:05:32.038 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-06-23T18:05:32.039 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc accepts -g... (cached) yes 2023-06-23T18:05:32.040 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-06-23T18:05:32.041 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gmake... no 2023-06-23T18:05:32.042 INFO:teuthology.orchestra.run.smithi022.stdout:checking for make... /usr/bin/make 2023-06-23T18:05:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:./configure: line 8824: test: $(SHELL): binary operator expected 2023-06-23T18:05:32.043 INFO:teuthology.orchestra.run.smithi022.stdout:checking for tar... /bin/tar 2023-06-23T18:05:32.044 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gzip... /bin/gzip 2023-06-23T18:05:32.062 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether gcc -MM is supported... yes 2023-06-23T18:05:32.062 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sort... /bin/sort 2023-06-23T18:05:32.063 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether ln -s works... yes 2023-06-23T18:05:32.064 INFO:teuthology.orchestra.run.smithi022.stdout:checking for rpm... no 2023-06-23T18:05:32.126 INFO:teuthology.orchestra.run.smithi022.stdout:checking assert.h usability... yes 2023-06-23T18:05:32.139 INFO:teuthology.orchestra.run.smithi022.stdout:checking assert.h presence... yes 2023-06-23T18:05:32.140 INFO:teuthology.orchestra.run.smithi022.stdout:checking for assert.h... yes 2023-06-23T18:05:32.197 INFO:teuthology.orchestra.run.smithi022.stdout:checking bstring.h usability... no 2023-06-23T18:05:32.218 INFO:teuthology.orchestra.run.smithi022.stdout:checking bstring.h presence... no 2023-06-23T18:05:32.219 INFO:teuthology.orchestra.run.smithi022.stdout:checking for bstring.h... no 2023-06-23T18:05:32.273 INFO:teuthology.orchestra.run.smithi022.stdout:checking libgen.h usability... yes 2023-06-23T18:05:32.288 INFO:teuthology.orchestra.run.smithi022.stdout:checking libgen.h presence... yes 2023-06-23T18:05:32.289 INFO:teuthology.orchestra.run.smithi022.stdout:checking for libgen.h... yes 2023-06-23T18:05:32.350 INFO:teuthology.orchestra.run.smithi022.stdout:checking dirent.h usability... yes 2023-06-23T18:05:32.367 INFO:teuthology.orchestra.run.smithi022.stdout:checking dirent.h presence... yes 2023-06-23T18:05:32.367 INFO:teuthology.orchestra.run.smithi022.stdout:checking for dirent.h... yes 2023-06-23T18:05:32.456 INFO:teuthology.orchestra.run.smithi022.stdout:checking errno.h usability... yes 2023-06-23T18:05:32.472 INFO:teuthology.orchestra.run.smithi022.stdout:checking errno.h presence... yes 2023-06-23T18:05:32.472 INFO:teuthology.orchestra.run.smithi022.stdout:checking for errno.h... yes 2023-06-23T18:05:32.527 INFO:teuthology.orchestra.run.smithi022.stdout:checking malloc.h usability... yes 2023-06-23T18:05:32.545 INFO:teuthology.orchestra.run.smithi022.stdout:checking malloc.h presence... yes 2023-06-23T18:05:32.546 INFO:teuthology.orchestra.run.smithi022.stdout:checking for malloc.h... yes 2023-06-23T18:05:32.601 INFO:teuthology.orchestra.run.smithi022.stdout:checking uuid.h usability... no 2023-06-23T18:05:32.622 INFO:teuthology.orchestra.run.smithi022.stdout:checking uuid.h presence... no 2023-06-23T18:05:32.623 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid.h... no 2023-06-23T18:05:32.681 INFO:teuthology.orchestra.run.smithi022.stdout:checking uuid/uuid.h usability... yes 2023-06-23T18:05:32.700 INFO:teuthology.orchestra.run.smithi022.stdout:checking uuid/uuid.h presence... yes 2023-06-23T18:05:32.700 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid/uuid.h... yes 2023-06-23T18:05:32.771 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/uuid.h usability... no 2023-06-23T18:05:32.791 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/uuid.h presence... no 2023-06-23T18:05:32.792 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/uuid.h... no 2023-06-23T18:05:32.847 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/file.h usability... yes 2023-06-23T18:05:32.860 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/file.h presence... yes 2023-06-23T18:05:32.860 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/file.h... yes 2023-06-23T18:05:32.919 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fcntl.h usability... yes 2023-06-23T18:05:32.934 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fcntl.h presence... yes 2023-06-23T18:05:32.934 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fcntl.h... yes 2023-06-23T18:05:32.990 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/syssgi.h usability... no 2023-06-23T18:05:33.009 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/syssgi.h presence... no 2023-06-23T18:05:33.010 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/syssgi.h... no 2023-06-23T18:05:33.104 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/param.h usability... yes 2023-06-23T18:05:33.128 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/param.h presence... yes 2023-06-23T18:05:33.128 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/param.h... yes 2023-06-23T18:05:33.139 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/stat.h... (cached) yes 2023-06-23T18:05:33.199 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/statvfs.h usability... yes 2023-06-23T18:05:33.215 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/statvfs.h presence... yes 2023-06-23T18:05:33.215 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/statvfs.h... yes 2023-06-23T18:05:33.275 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/time.h usability... yes 2023-06-23T18:05:33.291 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/time.h presence... yes 2023-06-23T18:05:33.292 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/time.h... yes 2023-06-23T18:05:33.352 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/ioctl.h usability... yes 2023-06-23T18:05:33.366 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/ioctl.h presence... yes 2023-06-23T18:05:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/ioctl.h... yes 2023-06-23T18:05:33.426 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/wait.h usability... yes 2023-06-23T18:05:33.444 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/wait.h presence... yes 2023-06-23T18:05:33.445 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/wait.h... yes 2023-06-23T18:05:33.457 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/types.h... (cached) yes 2023-06-23T18:05:33.469 INFO:teuthology.orchestra.run.smithi022.stdout:checking for strings.h... (cached) yes 2023-06-23T18:05:33.527 INFO:teuthology.orchestra.run.smithi022.stdout:checking err.h usability... yes 2023-06-23T18:05:33.542 INFO:teuthology.orchestra.run.smithi022.stdout:checking err.h presence... yes 2023-06-23T18:05:33.543 INFO:teuthology.orchestra.run.smithi022.stdout:checking for err.h... yes 2023-06-23T18:05:33.601 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/falloc.h usability... yes 2023-06-23T18:05:33.612 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/falloc.h presence... yes 2023-06-23T18:05:33.613 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/falloc.h... yes 2023-06-23T18:05:33.670 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-06-23T18:05:33.689 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-06-23T18:05:33.689 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fs/xfs_fsops.h... no 2023-06-23T18:05:33.742 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fs/xfs_itable.h usability... no 2023-06-23T18:05:33.762 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/fs/xfs_itable.h presence... no 2023-06-23T18:05:33.763 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/fs/xfs_itable.h... no 2023-06-23T18:05:33.820 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfs/platform_defs.h usability... no 2023-06-23T18:05:33.841 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfs/platform_defs.h presence... no 2023-06-23T18:05:33.842 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/platform_defs.h... no 2023-06-23T18:05:33.892 INFO:teuthology.orchestra.run.smithi022.stdout:checking btrfs/ioctl.h usability... no 2023-06-23T18:05:33.913 INFO:teuthology.orchestra.run.smithi022.stdout:checking btrfs/ioctl.h presence... no 2023-06-23T18:05:33.914 INFO:teuthology.orchestra.run.smithi022.stdout:checking for btrfs/ioctl.h... no 2023-06-23T18:05:33.963 INFO:teuthology.orchestra.run.smithi022.stdout:checking cifs/ioctl.h usability... no 2023-06-23T18:05:33.984 INFO:teuthology.orchestra.run.smithi022.stdout:checking cifs/ioctl.h presence... no 2023-06-23T18:05:33.984 INFO:teuthology.orchestra.run.smithi022.stdout:checking for cifs/ioctl.h... no 2023-06-23T18:05:34.039 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/mman.h usability... yes 2023-06-23T18:05:34.053 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/mman.h presence... yes 2023-06-23T18:05:34.053 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/mman.h... yes 2023-06-23T18:05:34.086 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xfs_log_format.h... no 2023-06-23T18:05:34.149 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid_compare... no 2023-06-23T18:05:34.202 INFO:teuthology.orchestra.run.smithi022.stdout:checking for uuid_compare in -luuid... yes 2023-06-23T18:05:34.304 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xfs.h... yes 2023-06-23T18:05:34.333 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/libxfs.h... no 2023-06-23T18:05:34.370 INFO:teuthology.orchestra.run.smithi022.stdout:checking whether xlog_assign_lsn is declared... no 2023-06-23T18:05:34.422 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/xqm.h... yes 2023-06-23T18:05:34.503 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfsctl from xfs/xfs.h... ok 2023-06-23T18:05:34.550 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfs/handle.h usability... yes 2023-06-23T18:05:34.563 INFO:teuthology.orchestra.run.smithi022.stdout:checking xfs/handle.h presence... yes 2023-06-23T18:05:34.563 INFO:teuthology.orchestra.run.smithi022.stdout:checking for xfs/handle.h... yes 2023-06-23T18:05:34.638 INFO:teuthology.orchestra.run.smithi022.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-06-23T18:05:34.702 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/acl.h usability... yes 2023-06-23T18:05:34.720 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/acl.h presence... yes 2023-06-23T18:05:34.721 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/acl.h... yes 2023-06-23T18:05:34.776 INFO:teuthology.orchestra.run.smithi022.stdout:checking acl/libacl.h usability... yes 2023-06-23T18:05:34.794 INFO:teuthology.orchestra.run.smithi022.stdout:checking acl/libacl.h presence... yes 2023-06-23T18:05:34.794 INFO:teuthology.orchestra.run.smithi022.stdout:checking for acl/libacl.h... yes 2023-06-23T18:05:34.857 INFO:teuthology.orchestra.run.smithi022.stdout:checking for acl_init in -lacl... yes 2023-06-23T18:05:34.904 INFO:teuthology.orchestra.run.smithi022.stdout:checking gdbm-ndbm.h usability... no 2023-06-23T18:05:34.924 INFO:teuthology.orchestra.run.smithi022.stdout:checking gdbm-ndbm.h presence... no 2023-06-23T18:05:34.925 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gdbm-ndbm.h... no 2023-06-23T18:05:34.970 INFO:teuthology.orchestra.run.smithi022.stdout:checking gdbm/ndbm.h usability... no 2023-06-23T18:05:34.990 INFO:teuthology.orchestra.run.smithi022.stdout:checking gdbm/ndbm.h presence... no 2023-06-23T18:05:34.990 INFO:teuthology.orchestra.run.smithi022.stdout:checking for gdbm/ndbm.h... no 2023-06-23T18:05:35.036 INFO:teuthology.orchestra.run.smithi022.stdout:checking ndbm.h usability... no 2023-06-23T18:05:35.058 INFO:teuthology.orchestra.run.smithi022.stdout:checking ndbm.h presence... no 2023-06-23T18:05:35.058 INFO:teuthology.orchestra.run.smithi022.stdout:checking for ndbm.h... no 2023-06-23T18:05:35.110 INFO:teuthology.orchestra.run.smithi022.stdout:checking libaio.h usability... yes 2023-06-23T18:05:35.133 INFO:teuthology.orchestra.run.smithi022.stdout:checking libaio.h presence... yes 2023-06-23T18:05:35.133 INFO:teuthology.orchestra.run.smithi022.stdout:checking for libaio.h... yes 2023-06-23T18:05:35.181 INFO:teuthology.orchestra.run.smithi022.stdout:checking liburing.h usability... no 2023-06-23T18:05:35.202 INFO:teuthology.orchestra.run.smithi022.stdout:checking liburing.h presence... no 2023-06-23T18:05:35.203 INFO:teuthology.orchestra.run.smithi022.stdout:checking for liburing.h... no 2023-06-23T18:05:35.246 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/capability.h usability... no 2023-06-23T18:05:35.265 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/capability.h presence... no 2023-06-23T18:05:35.266 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/capability.h... no 2023-06-23T18:05:35.316 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/fiemap.h usability... yes 2023-06-23T18:05:35.331 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/fiemap.h presence... yes 2023-06-23T18:05:35.332 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/fiemap.h... yes 2023-06-23T18:05:35.396 INFO:teuthology.orchestra.run.smithi022.stdout:checking for fallocate... yes 2023-06-23T18:05:35.460 INFO:teuthology.orchestra.run.smithi022.stdout:checking for open_by_handle_at... yes 2023-06-23T18:05:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/prctl.h usability... yes 2023-06-23T18:05:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:checking sys/prctl.h presence... yes 2023-06-23T18:05:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:checking for sys/prctl.h... yes 2023-06-23T18:05:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/fs.h usability... yes 2023-06-23T18:05:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/fs.h presence... yes 2023-06-23T18:05:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/fs.h... yes 2023-06-23T18:05:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:checking btrfsutil.h usability... no 2023-06-23T18:05:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:checking btrfsutil.h presence... no 2023-06-23T18:05:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:checking for btrfsutil.h... no 2023-06-23T18:05:35.719 INFO:teuthology.orchestra.run.smithi022.stdout:checking for copy_file_range... yes 2023-06-23T18:05:35.784 INFO:teuthology.orchestra.run.smithi022.stdout:checking for SEEK_DATA... yes 2023-06-23T18:05:35.860 INFO:teuthology.orchestra.run.smithi022.stdout:checking for BMV_OF_SHARED... yes 2023-06-23T18:05:35.913 INFO:teuthology.orchestra.run.smithi022.stdout:checking for nftw... yes 2023-06-23T18:05:35.975 INFO:teuthology.orchestra.run.smithi022.stdout:checking for RLIMIT_NOFILE... yes 2023-06-23T18:05:36.010 INFO:teuthology.orchestra.run.smithi022.stdout:checking for FIEXCHANGE_RANGE... no 2023-06-23T18:05:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:checking for renameat2... yes 2023-06-23T18:05:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:checking for reallocarray... yes 2023-06-23T18:05:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct mount_attr... no 2023-06-23T18:05:36.243 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_qgroup_limit... yes 2023-06-23T18:05:36.312 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-06-23T18:05:36.381 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-06-23T18:05:36.443 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-06-23T18:05:36.511 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-06-23T18:05:36.577 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-06-23T18:05:36.647 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-06-23T18:05:36.702 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/btrfs.h usability... yes 2023-06-23T18:05:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/btrfs.h presence... yes 2023-06-23T18:05:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/btrfs.h... yes 2023-06-23T18:05:36.775 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/btrfs_tree.h usability... yes 2023-06-23T18:05:36.789 INFO:teuthology.orchestra.run.smithi022.stdout:checking linux/btrfs_tree.h presence... yes 2023-06-23T18:05:36.789 INFO:teuthology.orchestra.run.smithi022.stdout:checking for linux/btrfs_tree.h... yes 2023-06-23T18:05:36.860 INFO:teuthology.orchestra.run.smithi022.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-06-23T18:05:36.893 INFO:teuthology.orchestra.run.smithi022.stdout:configure: creating ./config.status 2023-06-23T18:05:37.729 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: creating include/builddefs 2023-06-23T18:05:37.757 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: creating include/config.h 2023-06-23T18:05:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:config.status: executing libtool commands 2023-06-23T18:05:37.867 INFO:teuthology.orchestra.run.smithi022.stdout:Building include 2023-06-23T18:05:37.875 INFO:teuthology.orchestra.run.smithi022.stdout:Building lib 2023-06-23T18:05:37.882 INFO:teuthology.orchestra.run.smithi022.stdout: [LTDEP] 2023-06-23T18:05:38.078 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dataascii.lo 2023-06-23T18:05:38.255 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] databin.lo 2023-06-23T18:05:38.468 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] datapid.lo 2023-06-23T18:05:38.625 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] file_lock.lo 2023-06-23T18:05:38.854 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] forker.lo 2023-06-23T18:05:39.059 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] pattern.lo 2023-06-23T18:05:39.243 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] open_flags.lo 2023-06-23T18:05:39.453 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] random_range.lo 2023-06-23T18:05:39.768 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] string_to_tokens.lo 2023-06-23T18:05:39.924 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] str_to_bytes.lo 2023-06-23T18:05:40.111 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] tlibio.lo 2023-06-23T18:05:40.432 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] write_log.lo 2023-06-23T18:05:40.535 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c: In function 'wlog_record_write': 2023-06-23T18:05:40.535 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c:205:6: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:40.535 INFO:teuthology.orchestra.run.smithi022.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2023-06-23T18:05:40.536 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:40.536 INFO:teuthology.orchestra.run.smithi022.stderr:write_log.c:209:6: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:40.536 INFO:teuthology.orchestra.run.smithi022.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2023-06-23T18:05:40.536 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:40.741 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] random.lo 2023-06-23T18:05:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: [LD] 2023-06-23T18:05:41.020 INFO:teuthology.orchestra.run.smithi022.stdout:Building ltp 2023-06-23T18:05:41.026 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:41.249 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] doio 2023-06-23T18:05:42.060 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsstress 2023-06-23T18:05:43.591 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsx 2023-06-23T18:05:44.610 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] growfiles 2023-06-23T18:05:45.234 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] iogen 2023-06-23T18:05:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c: In function 'main': 2023-06-23T18:05:45.411 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c:280:2: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:45.411 INFO:teuthology.orchestra.run.smithi022.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2023-06-23T18:05:45.411 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c: In function 'parse_cmdline': 2023-06-23T18:05:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:iogen.c:1535:10: warning: ignoring return value of 'getcwd', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:45.415 INFO:teuthology.orchestra.run.smithi022.stderr: 1535 | getcwd(fptr->f_path, 2023-06-23T18:05:45.415 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:45.415 INFO:teuthology.orchestra.run.smithi022.stderr: 1536 | sizeof(fptr->f_path)-1); 2023-06-23T18:05:45.415 INFO:teuthology.orchestra.run.smithi022.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:45.684 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-stress 2023-06-23T18:05:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:aio-stress.c: In function 'setup_shared_mem': 2023-06-23T18:05:45.862 INFO:teuthology.orchestra.run.smithi022.stderr:aio-stress.c:1013:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:45.862 INFO:teuthology.orchestra.run.smithi022.stderr: 1013 | ftruncate(fd, total_ram); 2023-06-23T18:05:45.863 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:46.215 INFO:teuthology.orchestra.run.smithi022.stdout:Building src 2023-06-23T18:05:46.226 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:48.665 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:48.718 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:48.734 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:49.076 INFO:teuthology.orchestra.run.smithi022.stdout: [DEP] 2023-06-23T18:05:49.118 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirstress 2023-06-23T18:05:49.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:05:49.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:05:49.472 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fill 2023-06-23T18:05:49.764 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fill2 2023-06-23T18:05:50.073 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] getpagesize 2023-06-23T18:05:50.258 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] holes 2023-06-23T18:05:50.497 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] lstat64 2023-06-23T18:05:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] nametest 2023-06-23T18:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c: In function 'main': 2023-06-23T18:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:236:5: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr: 236 | write(1, "\n", 1); 2023-06-23T18:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:238:4: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr: 238 | write(1, ".", 1); 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:nametest.c:288:5: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr: 288 | write(1, ".", 1); 2023-06-23T18:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:05:51.070 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] permname 2023-06-23T18:05:51.352 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] randholes 2023-06-23T18:05:51.685 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] runas 2023-06-23T18:05:51.942 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] truncfile 2023-06-23T18:05:52.200 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] usemem 2023-06-23T18:05:52.421 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmapcat 2023-06-23T18:05:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:mmapcat.c: In function 'main': 2023-06-23T18:05:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:mmapcat.c:29:2: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:52.595 INFO:teuthology.orchestra.run.smithi022.stderr: 29 | write(1,ptr2,ptr - ptr2); 2023-06-23T18:05:52.595 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:52.647 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] append_reader 2023-06-23T18:05:52.885 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] append_writer 2023-06-23T18:05:53.115 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirperf 2023-06-23T18:05:53.416 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] metaperf 2023-06-23T18:05:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'crfiles': 2023-06-23T18:05:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:166:4: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.586 INFO:teuthology.orchestra.run.smithi022.stderr: 166 | write(fd, buf, fsize); 2023-06-23T18:05:53.586 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 't_linkun': 2023-06-23T18:05:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:454:4: warning: ignoring return value of 'link', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.587 INFO:teuthology.orchestra.run.smithi022.stderr: 454 | link("a", *fnp); 2023-06-23T18:05:53.587 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~ 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 't_chown': 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:404:5: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr: 404 | chown(*fnp, 2, -1); 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:406:5: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr: 406 | chown(*fnp, 1, -1); 2023-06-23T18:05:53.588 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'i_chown': 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:291:3: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr: 291 | chown(*fnp, 1, -1); 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c: In function 'main': 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:metaperf.c:152:2: warning: ignoring return value of 'chdir', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr: 152 | chdir(".."); 2023-06-23T18:05:53.589 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~ 2023-06-23T18:05:53.750 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] devzero 2023-06-23T18:05:54.001 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] feature 2023-06-23T18:05:54.302 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] alloc 2023-06-23T18:05:54.483 INFO:teuthology.orchestra.run.smithi022.stderr:alloc.c: In function 'main': 2023-06-23T18:05:54.484 INFO:teuthology.orchestra.run.smithi022.stderr:alloc.c:247:3: warning: ignoring return value of 'mkstemp', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:54.484 INFO:teuthology.orchestra.run.smithi022.stderr: 247 | mkstemp(tmpfile); 2023-06-23T18:05:54.484 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:05:54.589 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fault 2023-06-23T18:05:54.821 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fstest 2023-06-23T18:05:55.278 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_access_root 2023-06-23T18:05:55.452 INFO:teuthology.orchestra.run.smithi022.stderr:t_access_root.c: In function 'main': 2023-06-23T18:05:55.452 INFO:teuthology.orchestra.run.smithi022.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:55.453 INFO:teuthology.orchestra.run.smithi022.stderr: 55 | system(cmd); 2023-06-23T18:05:55.453 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~ 2023-06-23T18:05:55.513 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] godown 2023-06-23T18:05:55.757 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] resvtest 2023-06-23T18:05:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] writemod 2023-06-23T18:05:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] writev_on_pagefault 2023-06-23T18:05:56.481 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] makeextents 2023-06-23T18:05:56.736 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] itrash 2023-06-23T18:05:56.966 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] rename 2023-06-23T18:05:57.175 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] multi_open_unlink 2023-06-23T18:05:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] unwritten_sync 2023-06-23T18:05:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] genhashnames 2023-06-23T18:05:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_holes 2023-06-23T18:05:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_writev 2023-06-23T18:05:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_truncate_cmtime 2023-06-23T18:05:58.602 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dirhash_collide 2023-06-23T18:05:58.862 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_rename_overwrite 2023-06-23T18:05:59.070 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] holetest 2023-06-23T18:05:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:holetest.c: In function 'pt_write_marker': 2023-06-23T18:05:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:holetest.c:140:3: warning: ignoring return value of 'pwrite', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:59.253 INFO:teuthology.orchestra.run.smithi022.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2023-06-23T18:05:59.254 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:59.380 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_truncate_self 2023-06-23T18:05:59.579 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] af_unix 2023-06-23T18:05:59.805 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_stale_pmd 2023-06-23T18:05:59.976 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c: In function 'main': 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:42:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr: 42 | ftruncate(fd, 0); 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:43:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr: 43 | ftruncate(fd, MiB(4)); 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:05:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_stale_pmd.c:55:2: warning: ignoring return value of 'pwrite', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:05:59.978 INFO:teuthology.orchestra.run.smithi022.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2023-06-23T18:05:59.978 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:00.033 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_cow_race 2023-06-23T18:06:00.204 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c: In function 'main': 2023-06-23T18:06:00.205 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c:61:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:00.205 INFO:teuthology.orchestra.run.smithi022.stderr: 61 | ftruncate(fd, 0); 2023-06-23T18:06:00.205 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:00.206 INFO:teuthology.orchestra.run.smithi022.stderr:t_mmap_cow_race.c:62:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:00.206 INFO:teuthology.orchestra.run.smithi022.stderr: 62 | ftruncate(fd, MiB(2)); 2023-06-23T18:06:00.206 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:00.273 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_fallocate 2023-06-23T18:06:00.492 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsync-err 2023-06-23T18:06:00.761 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_write_ro 2023-06-23T18:06:00.986 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ext4_dax_journal_corruption 2023-06-23T18:06:01.158 INFO:teuthology.orchestra.run.smithi022.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2023-06-23T18:06:01.158 INFO:teuthology.orchestra.run.smithi022.stderr:t_ext4_dax_journal_corruption.c:61:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:01.159 INFO:teuthology.orchestra.run.smithi022.stderr: 61 | ftruncate(fd, 0); 2023-06-23T18:06:01.159 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:01.222 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ext4_dax_inline_corruption 2023-06-23T18:06:01.457 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_ofd_locks 2023-06-23T18:06:01.745 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_collision 2023-06-23T18:06:02.031 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmap-write-concurrent 2023-06-23T18:06:02.277 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_get_file_time 2023-06-23T18:06:02.501 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_create_short_dirs 2023-06-23T18:06:02.748 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_create_long_dirs 2023-06-23T18:06:02.992 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_enospc 2023-06-23T18:06:03.261 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_writev_overlap 2023-06-23T18:06:03.512 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] checkpoint_journal 2023-06-23T18:06:03.741 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mmap-rw-fault 2023-06-23T18:06:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] allocstale 2023-06-23T18:06:04.238 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_cow_memory_failure 2023-06-23T18:06:04.489 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fake-dump-rootino 2023-06-23T18:06:04.711 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-buf-fault 2023-06-23T18:06:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] xfsctl 2023-06-23T18:06:05.221 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bstat 2023-06-23T18:06:05.517 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mtab 2023-06-23T18:06:05.691 INFO:teuthology.orchestra.run.smithi022.stderr:t_mtab.c: In function 'update_mtab': 2023-06-23T18:06:05.691 INFO:teuthology.orchestra.run.smithi022.stderr:t_mtab.c:256:3: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:05.691 INFO:teuthology.orchestra.run.smithi022.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2023-06-23T18:06:05.691 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:05.786 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] getdevicesize 2023-06-23T18:06:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] preallo_rw_pattern_reader 2023-06-23T18:06:06.188 INFO:teuthology.orchestra.run.smithi022.stderr:In file included from preallo_rw_pattern_reader.c:2: 2023-06-23T18:06:06.188 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c: In function 'main': 2023-06-23T18:06:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c:38:4: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:06.189 INFO:teuthology.orchestra.run.smithi022.stderr: 38 | read(fd, &x, 1048576); 2023-06-23T18:06:06.189 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] preallo_rw_pattern_writer 2023-06-23T18:06:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:In file included from preallo_rw_pattern_writer.c:2: 2023-06-23T18:06:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c: In function 'main': 2023-06-23T18:06:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:iopat.c:34:4: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:06.425 INFO:teuthology.orchestra.run.smithi022.stderr: 34 | write(fd, &x, 1048576); 2023-06-23T18:06:06.425 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] ftrunc 2023-06-23T18:06:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] trunc 2023-06-23T18:06:06.931 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fs_perms 2023-06-23T18:06:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'main': 2023-06-23T18:06:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:07.105 INFO:teuthology.orchestra.run.smithi022.stderr: 54 | system("rm -f test.file"); 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'testsetup': 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr: 67 | system("cp testx.file test.file"); 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr: 69 | chown("test.file",cuserId,cgroupId); 2023-06-23T18:06:07.106 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c: In function 'testfperm': 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr: 123 | seteuid(0); 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~ 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr: 124 | setegid(0); 2023-06-23T18:06:07.107 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~ 2023-06-23T18:06:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] testx 2023-06-23T18:06:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] looptest 2023-06-23T18:06:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] locktest 2023-06-23T18:06:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] unwritten_mmap 2023-06-23T18:06:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_unlink_test 2023-06-23T18:06:08.484 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c: In function 'main': 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c:106:4: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test.c:178:3: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr: 178 | system(fname); 2023-06-23T18:06:08.485 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~ 2023-06-23T18:06:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] deduperace 2023-06-23T18:06:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_unlink_test_modified 2023-06-23T18:06:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2023-06-23T18:06:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:83:6: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: 83 | write(fd, fname, sizeof(fname)); 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:139:6: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: 139 | write(fd, fname, sizeof(fname)); 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr:bulkstat_unlink_test_modified.c:185:2: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: 185 | system(fname); 2023-06-23T18:06:09.043 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~ 2023-06-23T18:06:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_offset 2023-06-23T18:06:09.361 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_futimens 2023-06-23T18:06:09.576 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_immutable 2023-06-23T18:06:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'create_test_area': 2023-06-23T18:06:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.779 INFO:teuthology.orchestra.run.smithi022.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2023-06-23T18:06:09.779 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.780 INFO:teuthology.orchestra.run.smithi022.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2023-06-23T18:06:09.780 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'remove_test_area': 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2023-06-23T18:06:09.782 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.783 INFO:teuthology.orchestra.run.smithi022.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'check_test_area': 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.784 INFO:teuthology.orchestra.run.smithi022.stderr: 1788 | asprintf(&path, "%s/", dir); 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2023-06-23T18:06:09.786 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.788 INFO:teuthology.orchestra.run.smithi022.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2023-06-23T18:06:09.788 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'test_immutable': 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.790 INFO:teuthology.orchestra.run.smithi022.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:408:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr: 408 | chown(path, 0, 0); 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.791 INFO:teuthology.orchestra.run.smithi022.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:553:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr: 553 | chown(path, 0, 0); 2023-06-23T18:06:09.792 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.793 INFO:teuthology.orchestra.run.smithi022.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2023-06-23T18:06:09.794 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr: 653 | chown(path, 0, 0); 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.795 INFO:teuthology.orchestra.run.smithi022.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:705:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr: 705 | chown(path, 0, 0); 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.796 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c: In function 'test_append': 2023-06-23T18:06:09.797 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.797 INFO:teuthology.orchestra.run.smithi022.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2023-06-23T18:06:09.797 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.797 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1051:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr: 1051 | chown(path, 0, 0); 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.798 INFO:teuthology.orchestra.run.smithi022.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2023-06-23T18:06:09.800 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.800 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.800 INFO:teuthology.orchestra.run.smithi022.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1516:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr: 1516 | chown(path, 0, 0); 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.801 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.802 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1573:4: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.803 INFO:teuthology.orchestra.run.smithi022.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr: 1618 | chown(path, 0, 0); 2023-06-23T18:06:09.804 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr: 1644 | chown(path, 0, 0); 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.805 INFO:teuthology.orchestra.run.smithi022.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1692:9: warning: ignoring return value of 'chown', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr: 1692 | chown(path, 0, 0); 2023-06-23T18:06:09.806 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~ 2023-06-23T18:06:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:09.807 INFO:teuthology.orchestra.run.smithi022.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2023-06-23T18:06:09.807 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:10.383 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] stale_handle 2023-06-23T18:06:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:stale_handle.c: In function 'main': 2023-06-23T18:06:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:stale_handle.c:102:2: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:10.563 INFO:teuthology.orchestra.run.smithi022.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2023-06-23T18:06:10.563 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:10.626 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] pwrite_mmap_blocked 2023-06-23T18:06:10.845 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_offset2 2023-06-23T18:06:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] seek_sanity_test 2023-06-23T18:06:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'test18': 2023-06-23T18:06:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:460:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.313 INFO:teuthology.orchestra.run.smithi022.stderr: 460 | ftruncate(fd, 0); 2023-06-23T18:06:11.313 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'test_basic_support': 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:1207:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr: 1207 | ftruncate(fd, 0); 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:1256:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.315 INFO:teuthology.orchestra.run.smithi022.stderr: 1256 | ftruncate(fd, 0); 2023-06-23T18:06:11.316 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2023-06-23T18:06:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:88:3: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.316 INFO:teuthology.orchestra.run.smithi022.stderr: 88 | ftruncate(fd, 0); 2023-06-23T18:06:11.316 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:89:3: warning: ignoring return value of 'pwrite', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr: 89 | pwrite(fd, "a", 1, offset); 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:98:3: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr: 98 | ftruncate(fd, 0); 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr:seek_sanity_test.c:99:3: warning: ignoring return value of 'pwrite', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:11.317 INFO:teuthology.orchestra.run.smithi022.stderr: 99 | pwrite(fd, "a", 1, offset); 2023-06-23T18:06:11.318 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:11.652 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] seek_copy_test 2023-06-23T18:06:11.913 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_1 2023-06-23T18:06:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:t_readdir_1.c: In function 'main': 2023-06-23T18:06:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:12.082 INFO:teuthology.orchestra.run.smithi022.stderr: 30 | read(fd, buf, 100); 2023-06-23T18:06:12.082 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:12.129 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_2 2023-06-23T18:06:12.349 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fsync-tester 2023-06-23T18:06:12.527 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c: In function 'main': 2023-06-23T18:06:12.527 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:391:3: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:12.527 INFO:teuthology.orchestra.run.smithi022.stderr: 391 | read(fd, &seed, sizeof(seed)); 2023-06-23T18:06:12.527 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:12.527 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:541:3: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:12.528 INFO:teuthology.orchestra.run.smithi022.stderr: 541 | system("reboot -fn"); 2023-06-23T18:06:12.528 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:12.528 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c: In function 'drop_all_caches': 2023-06-23T18:06:12.528 INFO:teuthology.orchestra.run.smithi022.stderr:fsync-tester.c:90:2: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:12.528 INFO:teuthology.orchestra.run.smithi022.stderr: 90 | write(fd, value, sizeof(value)-1); 2023-06-23T18:06:12.529 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:12.691 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] nsexec 2023-06-23T18:06:12.942 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] cloner 2023-06-23T18:06:13.199 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] renameat2 2023-06-23T18:06:13.438 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_getcwd 2023-06-23T18:06:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:t_getcwd.c: In function 'test_getcwd': 2023-06-23T18:06:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:t_getcwd.c:21:3: warning: ignoring return value of 'getcwd', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:13.608 INFO:teuthology.orchestra.run.smithi022.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2023-06-23T18:06:13.609 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] e4compact 2023-06-23T18:06:13.960 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] test-nextquota 2023-06-23T18:06:14.209 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] punch-alternating 2023-06-23T18:06:14.464 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] attr-list-by-handle-cursor-test 2023-06-23T18:06:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] listxattr 2023-06-23T18:06:14.920 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-interleaved 2023-06-23T18:06:15.164 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_dir_type 2023-06-23T18:06:15.396 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] dio-invalidate-cache 2023-06-23T18:06:15.569 INFO:teuthology.orchestra.run.smithi022.stderr:dio-invalidate-cache.c: In function 'main': 2023-06-23T18:06:15.570 INFO:teuthology.orchestra.run.smithi022.stderr:dio-invalidate-cache.c:286:3: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:15.570 INFO:teuthology.orchestra.run.smithi022.stderr: 286 | write(fd, buf, blksz * nr_child); 2023-06-23T18:06:15.570 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:15.685 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] stat_test 2023-06-23T18:06:16.074 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_encrypted_d_revalidate 2023-06-23T18:06:16.316 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] attr_replace_test 2023-06-23T18:06:16.564 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] swapon 2023-06-23T18:06:16.811 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mkswap 2023-06-23T18:06:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_attr_corruption 2023-06-23T18:06:17.271 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_open_tmpfiles 2023-06-23T18:06:17.525 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fscrypt-crypt-util 2023-06-23T18:06:18.323 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] bulkstat_null_ocount 2023-06-23T18:06:18.549 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] splice-test 2023-06-23T18:06:18.812 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] chprojid_fail 2023-06-23T18:06:19.043 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] detached_mounts_propagation 2023-06-23T18:06:19.309 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] ext4_resize 2023-06-23T18:06:19.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:06:19.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:06:19.533 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_readdir_3 2023-06-23T18:06:19.801 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] splice2pipe 2023-06-23T18:06:19.969 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c: In function 'prepare_pipe': 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c:57:3: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr: 57 | write(p[1], buffer, n); 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:splice2pipe.c:65:3: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr: 65 | read(p[0], buffer, n); 2023-06-23T18:06:19.970 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:20.038 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] uuid_ioctl 2023-06-23T18:06:20.261 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fiemap-tester 2023-06-23T18:06:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_stripealign 2023-06-23T18:06:20.844 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] open_by_handle 2023-06-23T18:06:21.179 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] t_mmap_dio 2023-06-23T18:06:21.416 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] xfsfind 2023-06-23T18:06:21.686 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] fssum 2023-06-23T18:06:22.240 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2023-06-23T18:06:22.252 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] replay-log 2023-06-23T18:06:22.682 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2023-06-23T18:06:22.686 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2023-06-23T18:06:22.697 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-append-write-fallocate-race 2023-06-23T18:06:22.952 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-fcntl-race 2023-06-23T18:06:23.195 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-subblock-eof-read 2023-06-23T18:06:23.418 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-append-write-read-race 2023-06-23T18:06:23.677 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-invalidate-readahead 2023-06-23T18:06:23.928 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-hole-filling-race 2023-06-23T18:06:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-06-23T18:06:24.364 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-last-ref-held-by-io 2023-06-23T18:06:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-cycle-write 2023-06-23T18:06:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:aio-dio-cycle-write.c: In function 'init_test': 2023-06-23T18:06:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:aio-dio-cycle-write.c:83:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:24.792 INFO:teuthology.orchestra.run.smithi022.stderr: 83 | ftruncate(fd, buf_size); 2023-06-23T18:06:24.792 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-extend-stat 2023-06-23T18:06:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-06-23T18:06:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aiocp 2023-06-23T18:06:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-invalidate-failure 2023-06-23T18:06:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aiodio_sparse2 2023-06-23T18:06:26.087 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2023-06-23T18:06:26.087 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c:111:3: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:26.087 INFO:teuthology.orchestra.run.smithi022.stderr: 111 | ftruncate(fd, filesize); 2023-06-23T18:06:26.087 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:26.088 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2023-06-23T18:06:26.088 INFO:teuthology.orchestra.run.smithi022.stderr:aiodio_sparse2.c:274:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result [-Wunused-result] 2023-06-23T18:06:26.088 INFO:teuthology.orchestra.run.smithi022.stderr: 274 | ftruncate(fd, size); 2023-06-23T18:06:26.088 INFO:teuthology.orchestra.run.smithi022.stderr: | ^~~~~~~~~~~~~~~~~~~ 2023-06-23T18:06:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-cow-race 2023-06-23T18:06:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-write-verify 2023-06-23T18:06:26.754 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] aio-dio-eof-race 2023-06-23T18:06:27.028 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2023-06-23T18:06:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] vfstest 2023-06-23T18:06:29.370 INFO:teuthology.orchestra.run.smithi022.stdout: [CC] mount-idmapped 2023-06-23T18:06:29.848 INFO:teuthology.orchestra.run.smithi022.stdout:Building m4 2023-06-23T18:06:29.852 INFO:teuthology.orchestra.run.smithi022.stdout:Building common 2023-06-23T18:06:29.859 INFO:teuthology.orchestra.run.smithi022.stdout:Building tests 2023-06-23T18:06:29.867 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2023-06-23T18:06:29.879 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/perf/group.list 2023-06-23T18:06:29.902 INFO:teuthology.orchestra.run.smithi022.stdout:Building ext4 2023-06-23T18:06:29.913 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ext4/group.list 2023-06-23T18:06:29.990 INFO:teuthology.orchestra.run.smithi022.stdout:Building ocfs2 2023-06-23T18:06:30.001 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ocfs2/group.list 2023-06-23T18:06:30.031 INFO:teuthology.orchestra.run.smithi022.stdout:Building overlay 2023-06-23T18:06:30.043 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/overlay/group.list 2023-06-23T18:06:30.122 INFO:teuthology.orchestra.run.smithi022.stdout:Building cifs 2023-06-23T18:06:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/cifs/group.list 2023-06-23T18:06:30.161 INFO:teuthology.orchestra.run.smithi022.stdout:Building generic 2023-06-23T18:06:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/generic/group.list 2023-06-23T18:06:30.374 INFO:teuthology.orchestra.run.smithi022.stdout:Building shared 2023-06-23T18:06:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/shared/group.list 2023-06-23T18:06:30.420 INFO:teuthology.orchestra.run.smithi022.stdout:Building xfs 2023-06-23T18:06:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/xfs/group.list 2023-06-23T18:06:30.632 INFO:teuthology.orchestra.run.smithi022.stdout:Building f2fs 2023-06-23T18:06:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/f2fs/group.list 2023-06-23T18:06:30.678 INFO:teuthology.orchestra.run.smithi022.stdout:Building udf 2023-06-23T18:06:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/udf/group.list 2023-06-23T18:06:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:Building ceph 2023-06-23T18:06:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ceph/group.list 2023-06-23T18:06:30.749 INFO:teuthology.orchestra.run.smithi022.stdout:Building nfs 2023-06-23T18:06:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/nfs/group.list 2023-06-23T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stdout:Building btrfs 2023-06-23T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/btrfs/group.list 2023-06-23T18:06:30.922 INFO:teuthology.orchestra.run.smithi022.stdout:Building tmpfs 2023-06-23T18:06:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/tmpfs/group.list 2023-06-23T18:06:30.961 INFO:teuthology.orchestra.run.smithi022.stdout:Building selftest 2023-06-23T18:06:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/selftest/group.list 2023-06-23T18:06:31.001 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo make install) 2023-06-23T18:06:31.043 INFO:teuthology.orchestra.run.smithi022.stdout:Building include 2023-06-23T18:06:31.050 INFO:teuthology.orchestra.run.smithi022.stdout:Building lib 2023-06-23T18:06:31.058 INFO:teuthology.orchestra.run.smithi022.stdout:Building ltp 2023-06-23T18:06:31.067 INFO:teuthology.orchestra.run.smithi022.stdout:Building src 2023-06-23T18:06:31.136 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2023-06-23T18:06:31.144 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2023-06-23T18:06:31.150 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2023-06-23T18:06:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2023-06-23T18:06:31.167 INFO:teuthology.orchestra.run.smithi022.stdout:Building m4 2023-06-23T18:06:31.175 INFO:teuthology.orchestra.run.smithi022.stdout:Building common 2023-06-23T18:06:31.182 INFO:teuthology.orchestra.run.smithi022.stdout:Building tests 2023-06-23T18:06:31.193 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2023-06-23T18:06:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/perf/group.list 2023-06-23T18:06:31.228 INFO:teuthology.orchestra.run.smithi022.stdout:Building ext4 2023-06-23T18:06:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ext4/group.list 2023-06-23T18:06:31.317 INFO:teuthology.orchestra.run.smithi022.stdout:Building ocfs2 2023-06-23T18:06:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ocfs2/group.list 2023-06-23T18:06:31.359 INFO:teuthology.orchestra.run.smithi022.stdout:Building overlay 2023-06-23T18:06:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/overlay/group.list 2023-06-23T18:06:31.448 INFO:teuthology.orchestra.run.smithi022.stdout:Building cifs 2023-06-23T18:06:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/cifs/group.list 2023-06-23T18:06:31.490 INFO:teuthology.orchestra.run.smithi022.stdout:Building generic 2023-06-23T18:06:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/generic/group.list 2023-06-23T18:06:31.698 INFO:teuthology.orchestra.run.smithi022.stdout:Building shared 2023-06-23T18:06:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/shared/group.list 2023-06-23T18:06:31.747 INFO:teuthology.orchestra.run.smithi022.stdout:Building xfs 2023-06-23T18:06:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/xfs/group.list 2023-06-23T18:06:31.961 INFO:teuthology.orchestra.run.smithi022.stdout:Building f2fs 2023-06-23T18:06:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/f2fs/group.list 2023-06-23T18:06:32.009 INFO:teuthology.orchestra.run.smithi022.stdout:Building udf 2023-06-23T18:06:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/udf/group.list 2023-06-23T18:06:32.047 INFO:teuthology.orchestra.run.smithi022.stdout:Building ceph 2023-06-23T18:06:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/ceph/group.list 2023-06-23T18:06:32.089 INFO:teuthology.orchestra.run.smithi022.stdout:Building nfs 2023-06-23T18:06:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/nfs/group.list 2023-06-23T18:06:32.134 INFO:teuthology.orchestra.run.smithi022.stdout:Building btrfs 2023-06-23T18:06:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/btrfs/group.list 2023-06-23T18:06:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:Building tmpfs 2023-06-23T18:06:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/tmpfs/group.list 2023-06-23T18:06:32.299 INFO:teuthology.orchestra.run.smithi022.stdout:Building selftest 2023-06-23T18:06:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: [GROUP] /tmp/tmp.420M2hxqAdxfstests-dev/tests/selftest/group.list 2023-06-23T18:06:32.337 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C include install 2023-06-23T18:06:32.341 INFO:teuthology.orchestra.run.smithi022.stdout:make[1]: Nothing to be done for 'install'. 2023-06-23T18:06:32.342 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C lib install 2023-06-23T18:06:32.348 INFO:teuthology.orchestra.run.smithi022.stdout:make[1]: Nothing to be done for 'install'. 2023-06-23T18:06:32.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C ltp install 2023-06-23T18:06:32.353 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-06-23T18:06:32.373 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-06-23T18:06:32.539 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-06-23T18:06:32.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C src install 2023-06-23T18:06:32.596 INFO:teuthology.orchestra.run.smithi022.stdout:Building log-writes 2023-06-23T18:06:32.604 INFO:teuthology.orchestra.run.smithi022.stdout:Building perf 2023-06-23T18:06:32.612 INFO:teuthology.orchestra.run.smithi022.stdout:Building aio-dio-regress 2023-06-23T18:06:32.620 INFO:teuthology.orchestra.run.smithi022.stdout:Building vfs 2023-06-23T18:06:32.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make -C log-writes install 2023-06-23T18:06:32.635 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-06-23T18:06:32.654 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-06-23T18:06:32.673 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make -C perf install 2023-06-23T18:06:32.679 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-06-23T18:06:32.697 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-06-23T18:06:32.739 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make -C aio-dio-regress install 2023-06-23T18:06:32.746 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-06-23T18:06:32.765 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aiocp aio-dio-append-write-fallocate-race aio-dio-invalidate-failure aiodio_sparse2 aio-dio-cow-race aio-dio-write-verify aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-06-23T18:06:32.876 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make -C vfs install 2023-06-23T18:06:32.882 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-06-23T18:06:32.901 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-06-23T18:06:32.926 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-06-23T18:06:32.946 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-06-23T18:06:35.745 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-06-23T18:06:35.958 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-06-23T18:06:36.014 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C m4 install 2023-06-23T18:06:36.020 INFO:teuthology.orchestra.run.smithi022.stdout:make[1]: Nothing to be done for 'install'. 2023-06-23T18:06:36.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C common install 2023-06-23T18:06:36.024 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-06-23T18:06:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-06-23T18:06:36.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C tests install 2023-06-23T18:06:36.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/btrfs/ install 2023-06-23T18:06:36.328 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-06-23T18:06:36.348 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 290 291 299 /var/lib/xfstests/tests/btrfs 2023-06-23T18:06:37.964 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-06-23T18:06:37.984 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 016.out 064.out 186.out 240.out 279.out 180.out 282.out 267.out 280.out 140.out 265.out 206.out 121.out 040.out 106.out 199.out 233.out 234.out 248.out 009.out 033.out 082.out 252.out 270.out 148.out 088.out 137.out 002.out 125.out 110.out 165.out 152.out 143.out 264.out 015.out 192.out 075.out 178.out 284.out 052.out 228.out 276.out 046.out 273.out 074.out 251.out 176.out 209.out 020.out 195.out 215.out 115.out 179.out 250.out 191.out 266.out 111.out 051.out 177.out 219.out 119.out 244.out 128.out 232.out 087.out 212.out 112.out 012.out 107.out 162.out 147.out 196.out 145.out 004.out 187.out 024.out 220.out 238.out 120.out 202.out 102.out 171.out 261.out 156.out 092.out 072.out 068.out 048.out 062.out 084.out 129.out 066.out 229.out 254.out 047.out 011.out 029.out 035.out 134.out 170.out 260.out 034.out 065.out 286.out 161.out 246.out 056.out 216.out 116.out 043.out 274.out 287.out 010.out 001.out 138.out 237.out 097.out 153.out 160.out 175.out 091.out 081.out 071.out 061.out 239.out 067.out 200.out 100.out 025.out 258.out 269.out 208.out 039.out 222.out 275.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 108.out 006.out 167.out 217.out 133.out 117.out 017.out 236.out 159.out 211.out 158.out 085.out 277.out 146.out 188.out 189.out 204.out 104.out 225.out 242.out 268.out 135.out 257.out 124.out 142.out 172.out 103.out 058.out 262.out 042.out 096.out 193.out 283.out 083.out 099.out 247.out 053.out 109.out 157.out 164.out 154.out 094.out 241.out 089.out 141.out 214.out 114.out 014.out 278.out 041.out 223.out 028.out 086.out 123.out 144.out 032.out 182.out 205.out 136.out 005.out 036.out 198.out 008.out 190.out 173.out 050.out 038.out 105.out 263.out 291.out 185.out 095.out 285.out 155.out 201.out 019.out 063.out 299.out 151.out 231.out 227.out 069.out 131.out 213.out 113.out 013.out 031.out 057.out 272.out 078.out 184.out 259.out 210.out 132.out 037.out 093.out 197.out 235.out 127.out 230.out 027.out 130.out 023.out 044.out 073.out 030.out 203.out 003.out 101.out 055.out 166.out 245.out 243.out 194.out 249.out 122.out 076.out 022.out 174.out 045.out 281.out 090.out 224.out 181.out 271.out 149.out 183.out 077.out 218.out 118.out 018.out 256.out 168.out 007.out 079.out 054.out 059.out 150.out 049.out 226.out 163.out 126.out 026.out 021.out 253.out 255.out 139.out 221.out /var/lib/xfstests/tests/btrfs 2023-06-23T18:06:39.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/ceph/ install 2023-06-23T18:06:39.606 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-06-23T18:06:39.626 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-06-23T18:06:39.664 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-06-23T18:06:39.684 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-06-23T18:06:39.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/cifs/ install 2023-06-23T18:06:39.725 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-06-23T18:06:39.746 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-06-23T18:06:39.761 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-06-23T18:06:39.780 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-06-23T18:06:39.799 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/ext4/ install 2023-06-23T18:06:39.804 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-06-23T18:06:39.824 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-06-23T18:06:40.192 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-06-23T18:06:40.212 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 049.out 032.out 039.out 045.out 054.out 056.out 308.out 008.out 040.out 307.out 005.out 021.out 012.out 009.out 033.out 015.out 050.out 026.out 034.out 046.out 018.out 010.out 047.out 303.out 302.out 020.out 002.out 028.out 024.out 013.out 031.out 027.out 306.out 048.out 038.out 044.out 052.out 025.out 029.out 011.out 042.out 304.out 055.out 030.out 003.out 017.out 043.out 053.out 035.out 016.out 058.out 022.out 006.out 004.out 057.out 014.out 036.out 271.out 051.out 041.out 037.out 305.out 019.out 301.out 007.out 023.out 059.out /var/lib/xfstests/tests/ext4 2023-06-23T18:06:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/f2fs/ install 2023-06-23T18:06:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-06-23T18:06:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-06-23T18:06:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-06-23T18:06:40.688 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-06-23T18:06:40.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/generic/ install 2023-06-23T18:06:40.721 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-06-23T18:06:40.740 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 /var/lib/xfstests/tests/generic 2023-06-23T18:06:44.760 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-06-23T18:06:44.778 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 035.out.nfs 035.out.default 050.out.nojournal 050.out.xfsquota 050.out.default 050.cfg 035.cfg 021.out 233.out 518.out 055.out 295.out 683.out 337.out 385.out 082.out 270.out 460.out 237.out 502.out 088.out 597.out 097.out 202.out 137.out 034.out 110.out 389.out 593.out 571.out 040.out 646.out 592.out 282.out 728.out 468.out 628.out 428.out 507.out 328.out 566.out 127.out 557.out 273.out 182.out 420.out 609.out 576.out 087.out 506.out 057.out 195.out 711.out 007.out 466.out 311.out 532.out 591.out 719.out 111.out 432.out 519.out 623.out 131.out 319.out 177.out 219.out 119.out 423.out 661.out 232.out 134.out 260.out 095.out 611.out 638.out 465.out 716.out 161.out 416.out 026.out 216.out 116.out 043.out 358.out 338.out 238.out 439.out 138.out 263.out 497.out 160.out 647.out 433.out 481.out 061.out 377.out 686.out 333.out 258.out 322.out 039.out 222.out 098.out 330.out 690.out 480.out 706.out 606.out 406.out 306.out 206.out 106.out 006.out 447.out 003.out 217.out 120.out 191.out 117.out 355.out 357.out 542.out 485.out 074.out 033.out 188.out 367.out 489.out 279.out 624.out 524.out 174.out 163.out 324.out 224.out 257.out 124.out 672.out 668.out 058.out 262.out 366.out 648.out 042.out 537.out 193.out 083.out 399.out 365.out 509.out 568.out 309.out 209.out 441.out 255.out 356.out 474.out 341.out 640.out 094.out 241.out 089.out 691.out 616.out 614.out 514.out 312.out 141.out 314.out 214.out 381.out 056.out 671.out 041.out 016.out 340.out 649.out 572.out 586.out 240.out 549.out 673.out 633.out 321.out 140.out 708.out 608.out 449.out 185.out 408.out 308.out 208.out 380.out 670.out 604.out 404.out 104.out 047.out 599.out 476.out 192.out 199.out 278.out 639.out 443.out 540.out 186.out 539.out 133.out 164.out 376.out 589.out 154.out 339.out 652.out 239.out 440.out 498.out 139.out 180.out 267.out 317.out 552.out 017.out 169.out 565.out 315.out 580.out 452.out 679.out 336.out 020.out 252.out 585.out 678.out 448.out 352.out 348.out 651.out 184.out 435.out 409.out 248.out 009.out 482.out 625.out 283.out 488.out 496.out 325.out 551.out 225.out 636.out 125.out 544.out 693.out 296.out 650.out 152.out 563.out 143.out 024.out 015.out 392.out 682.out 472.out 595.out 578.out 642.out 550.out 052.out 062.out 655.out 276.out 451.out 450.out 463.out 004.out 157.out 069.out 644.out 316.out 344.out 487.out 715.out 615.out 395.out 151.out 415.out 674.out 215.out 115.out 384.out 351.out 250.out 266.out 051.out 536.out 228.out 577.out 244.out 128.out 662.out 579.out 144.out 405.out 205.out 136.out 598.out 363.out 712.out 176.out 198.out 723.out 108.out 390.out 680.out 705.out 605.out 505.out 305.out 105.out 005.out 523.out 084.out 149.out 179.out 703.out 619.out 727.out 685.out 562.out 419.out 388.out 527.out 158.out 044.out 019.out 695.out 086.out 574.out 332.out 060.out 142.out 584.out 045.out 372.out 478.out 612.out 462.out 173.out 687.out 350.out 132.out 037.out 293.out 197.out 492.out 499.out 246.out 032.out 112.out 660.out 414.out 063.out 101.out 561.out 343.out 323.out 702.out 362.out 387.out 594.out 438.out 721.out 289.out 699.out 284.out 477.out 621.out 290.out 548.out 299.out 456.out 707.out 461.out 391.out 445.out 632.out 256.out 493.out 107.out 658.out 251.out 526.out 479.out 713.out 426.out 221.out 326.out 226.out 413.out 126.out 121.out 313.out 555.out 694.out 347.out 275.out 038.out 090.out 361.out 528.out 564.out 294.out 656.out 066.out 455.out 558.out 162.out 147.out 288.out 396.out 547.out 531.out 698.out 148.out 301.out 360.out 374.out 261.out 076.out 657.out 692.out 304.out 634.out 724.out 129.out 726.out 458.out 534.out 626.out 529.out 603.out 434.out 329.out 587.out 165.out 229.out 590.out 371.out 234.out 183.out 645.out 486.out 223.out 123.out 669.out 023.out 459.out 688.out 277.out 516.out 146.out 398.out 696.out 247.out 427.out 190.out 280.out 484.out 491.out 187.out 046.out 227.out 583.out 285.out 631.out 654.out 359.out 501.out 171.out 155.out 431.out 099.out 331.out 231.out 613.out 513.out 028.out 213.out 113.out 013.out 031.out 630.out 675.out 667.out 272.out 530.out 627.out 573.out 430.out 310.out 454.out 259.out 210.out 327.out 397.out 067.out 093.out 287.out 230.out 027.out 292.out 130.out 370.out 403.out 030.out 203.out 103.out 354.out 025.out 386.out 560.out 722.out 291.out 622.out 243.out 522.out 543.out 422.out 394.out 159.out 249.out 122.out 689.out 166.out 022.out 424.out 637.out 684.out 490.out 353.out 375.out 181.out 676.out 271.out 570.out 618.out 064.out 018.out 418.out 077.out 218.out 118.out 575.out 172.out 079.out 469.out 054.out 059.out 049.out 442.out 569.out 521.out 378.out 253.out 421.out 475.out 512.out 494.out 412.out 212.out 538.out 379.out 342.out 012.out 167.out 554.out 471.out 070.out 596.out 720.out 620.out 196.out 520.out 242.out 457.out 320.out 220.out 629.out 602.out 153.out 402.out 302.out 102.out 002.out 717.out 617.out 517.out 429.out 417.out 156.out 092.out 664.out 072.out 068.out 473.out 369.out 511.out 411.out 029.out 211.out 011.out 659.out 194.out 170.out 053.out 065.out 286.out 245.out 437.out 545.out 714.out 559.out 710.out 036.out 610.out 510.out 274.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 495.out 281.out 048.out 665.out 114.out 014.out 346.out 581.out 297.out 145.out 168.out 641.out 436.out 091.out 081.out 071.out 075.out 653.out 700.out 600.out 500.out 400.out 300.out 200.out 100.out 464.out 541.out 269.out 546.out 382.out 677.out 553.out 178.out 175.out 298.out 446.out 080.out 709.out 607.out 508.out 407.out 307.out 207.out 109.out 008.out 643.out 150.out 349.out 453.out 666.out 236.out 078.out 364.out 567.out 725.out 582.out 383.out 085.out 704.out 588.out 635.out 334.out 504.out 525.out 189.out 204.out 535.out 425.out 345.out 318.out 467.out 368.out 681.out 254.out 335.out 718.out 073.out 235.out 470.out 268.out 556.out 135.out 503.out 533.out 303.out 264.out 515.out 373.out 265.out 697.out 444.out 483.out 096.out 393.out 663.out /var/lib/xfstests/tests/generic 2023-06-23T18:06:48.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/nfs/ install 2023-06-23T18:06:48.852 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-06-23T18:06:48.873 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-06-23T18:06:48.892 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-06-23T18:06:48.912 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-06-23T18:06:48.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/ocfs2/ install 2023-06-23T18:06:48.938 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-06-23T18:06:48.958 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-06-23T18:06:48.979 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-06-23T18:06:48.999 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-06-23T18:06:49.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/overlay/ install 2023-06-23T18:06:49.025 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-06-23T18:06:49.045 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-06-23T18:06:49.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:06:49.572 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:06:49.583 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-06-23T18:06:49.602 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 050.out 044.out 019.out 103.out 003.out 074.out 018.out 024.out 078.out 069.out 020.out 102.out 002.out 060.out 113.out 013.out 031.out 015.out 117.out 045.out 072.out 106.out 048.out 062.out 038.out 052.out 076.out 037.out 029.out 111.out 112.out 042.out 035.out 073.out 068.out 030.out 034.out 017.out 055.out 053.out 109.out 116.out 043.out 056.out 010.out 001.out 009.out 065.out 022.out 004.out 115.out 066.out 114.out 014.out 071.out 007.out 061.out 051.out 011.out 041.out 067.out 058.out 100.out 064.out 104.out 028.out 027.out 023.out 059.out 049.out 032.out 039.out 077.out 105.out 005.out 025.out 026.out 036.out 070.out 108.out 008.out 040.out 107.out 006.out 075.out 063.out 021.out 016.out 047.out 110.out 046.out 012.out 101.out 054.out 033.out 057.out /var/lib/xfstests/tests/overlay 2023-06-23T18:06:50.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/perf/ install 2023-06-23T18:06:50.139 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-06-23T18:06:50.157 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-06-23T18:06:50.176 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-06-23T18:06:50.194 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-06-23T18:06:50.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/selftest/ install 2023-06-23T18:06:50.216 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-06-23T18:06:50.234 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-06-23T18:06:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-06-23T18:06:50.293 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 006.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-06-23T18:06:50.334 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/shared/ install 2023-06-23T18:06:50.340 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-06-23T18:06:50.357 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-06-23T18:06:50.383 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-06-23T18:06:50.401 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-06-23T18:06:50.428 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/tmpfs/ install 2023-06-23T18:06:50.433 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-06-23T18:06:50.450 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-06-23T18:06:50.465 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-06-23T18:06:50.483 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-06-23T18:06:50.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/udf/ install 2023-06-23T18:06:50.506 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-06-23T18:06:50.524 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-06-23T18:06:50.544 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-06-23T18:06:50.563 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-06-23T18:06:50.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/make --no-print-directory Q=@ -C /tmp/tmp.420M2hxqAdxfstests-dev/tests/xfs/ install 2023-06-23T18:06:50.592 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-06-23T18:06:50.612 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-06-23T18:06:54.428 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-06-23T18:06:54.449 INFO:teuthology.orchestra.run.smithi022.stdout:../../install-sh -o root -g root -m 644 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 096.out.internal 216.out.64mblog 071.out.32 216.cfg 033.cfg 508.out 221.out 057.out 308.out 208.out 737.out 108.out 404.out 457.out 021.out 455.out 233.out 194.out 443.out 295.out 337.out 237.out 759.out 502.out 088.out 366.out 202.out 137.out 002.out 389.out 593.out 571.out 775.out 040.out 765.out 592.out 282.out 728.out 528.out 428.out 407.out 328.out 566.out 261.out 557.out 273.out 468.out 074.out 427.out 059.out 595.out 789.out 195.out 346.out 732.out 784.out 532.out 191.out 111.out 432.out 519.out 419.out 131.out 319.out 177.out 219.out 119.out 423.out 232.out 329.out 467.out 229.out 103.out 234.out 183.out 733.out 134.out 260.out 157.out 718.out 034.out 465.out 716.out 161.out 416.out 116.out 043.out 155.out 358.out 338.out 238.out 065.out 439.out 138.out 189.out 097.out 160.out 175.out 373.out 481.out 345.out 025.out 772.out 768.out 258.out 379.out 322.out 039.out 498.out 742.out 290.out 480.out 060.out 506.out 406.out 306.out 206.out 006.out 167.out 003.out 217.out 117.out 355.out 017.out 542.out 783.out 485.out 442.out 188.out 099.out 342.out 279.out 524.out 174.out 771.out 324.out 224.out 313.out 257.out 124.out 758.out 058.out 741.out 384.out 262.out 042.out 497.out 377.out 193.out 799.out 473.out 770.out 365.out 409.out 309.out 209.out 441.out 109.out 255.out 740.out 558.out 474.out 341.out 094.out 555.out 712.out 241.out 089.out 540.out 256.out 714.out 514.out 414.out 314.out 214.out 114.out 381.out 147.out 275.out 041.out 721.out 340.out 793.out 156.out 572.out 568.out 586.out 240.out 180.out 549.out 267.out 280.out 543.out 140.out 708.out 449.out 185.out 334.out 504.out 525.out 204.out 535.out 104.out 425.out 399.out 254.out 335.out 531.out 235.out 470.out 268.out 556.out 135.out 503.out 533.out 303.out 264.out 110.out 318.out 515.out 433.out 036.out 169.out 483.out 393.out 333.out 476.out 087.out 192.out 199.out 739.out 278.out 591.out 752.out 736.out 236.out 539.out 749.out 164.out 376.out 154.out 587.out 339.out 239.out 440.out 458.out 456.out 489.out 139.out 774.out 790.out 317.out 552.out 781.out 064.out 444.out 580.out 452.out 020.out 252.out 724.out 352.out 794.out 348.out 709.out 394.out 356.out 184.out 435.out 509.out 466.out 248.out 785.out 009.out 283.out 750.out 045.out 297.out 488.out 496.out 325.out 551.out 225.out 125.out 186.out 544.out 296.out 152.out 563.out 143.out 436.out 392.out 545.out 075.out 472.out 463.out 550.out 052.out 062.out 276.out 266.out 450.out 744.out 004.out 764.out 069.out 316.out 487.out 762.out 715.out 395.out 151.out 415.out 315.out 344.out 215.out 115.out 015.out 351.out 250.out 464.out 051.out 536.out 228.out 244.out 128.out 385.out 336.out 086.out 579.out 144.out 505.out 205.out 136.out 537.out 176.out 198.out 723.out 390.out 178.out 050.out 405.out 305.out 105.out 005.out 523.out 761.out 437.out 719.out 727.out 363.out 095.out 562.out 382.out 271.out 570.out 388.out 590.out 527.out 158.out 044.out 019.out 577.out 777.out 760.out 574.out 332.out 372.out 478.out 462.out 220.out 350.out 132.out 293.out 492.out 263.out 212.out 499.out 246.out 112.out 063.out 101.out 561.out 343.out 362.out 387.out 798.out 594.out 438.out 265.out 289.out 106.out 477.out 281.out 548.out 299.out 364.out 461.out 391.out 451.out 493.out 786.out 007.out 448.out 251.out 380.out 526.out 479.out 713.out 426.out 150.out 326.out 082.out 226.out 413.out 126.out 026.out 347.out 035.out 361.out 564.out 294.out 788.out 066.out 795.out 076.out 747.out 585.out 162.out 796.out 288.out 197.out 396.out 576.out 547.out 298.out 148.out 301.out 360.out 374.out 734.out 447.out 292.out 304.out 482.out 129.out 726.out 729.out 048.out 534.out 073.out 529.out 434.out 368.out 486.out 223.out 123.out 179.out 023.out 032.out 748.out 277.out 516.out 424.out 398.out 168.out 751.out 190.out 173.out 484.out 491.out 754.out 016.out 046.out 227.out 731.out 782.out 583.out 285.out 127.out 501.out 171.out 581.out 027.out 431.out 068.out 331.out 791.out 743.out 047.out 231.out 024.out 513.out 371.out 213.out 730.out 113.out 013.out 031.out 367.out 182.out 323.out 272.out 530.out 573.out 430.out 454.out 259.out 210.out 327.out 330.out 067.out 093.out 287.out 230.out 769.out 061.out 130.out 370.out 403.out 030.out 203.out 354.out 165.out 386.out 560.out 055.out 722.out 291.out 245.out 243.out 522.out 787.out 745.out 422.out 222.out 159.out 249.out 122.out 166.out 022.out 038.out 578.out 490.out 565.out 353.out 375.out 181.out 500.out 149.out 518.out 018.out 418.out 077.out 218.out 118.out 575.out 780.out 172.out 792.out 079.out 469.out 054.out 049.out 247.out 738.out 163.out 776.out 521.out 270.out 378.out 253.out 421.out 321.out 475.out 512.out 494.out 412.out 312.out 121.out 538.out 012.out 028.out 554.out 397.out 773.out 720.out 196.out 520.out 242.out 420.out 098.out 320.out 120.out 153.out 402.out 302.out 102.out 717.out 460.out 517.out 429.out 417.out 142.out 092.out 357.out 072.out 778.out 084.out 056.out 369.out 711.out 511.out 411.out 029.out 311.out 211.out 011.out 767.out 187.out 170.out 053.out 133.out 286.out 756.out 755.out 766.out 559.out 710.out 510.out 274.out 410.out 310.out 401.out 201.out 010.out 001.out 141.out 495.out 584.out 589.out 753.out 014.out 459.out 145.out 797.out 757.out 091.out 081.out 083.out 800.out 037.out 400.out 300.out 200.out 100.out 359.out 471.out 541.out 445.out 779.out 269.out 546.out 553.out 763.out 446.out 090.out 080.out 070.out 507.out 408.out 307.out 207.out 107.out 008.out 746.out 284.out 349.out 453.out 078.out 567.out 725.out 582.out 735.out 383.out 085.out 588.out 146.out /var/lib/xfstests/tests/xfs 2023-06-23T18:06:58.273 INFO:teuthology.orchestra.run.smithi022.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-06-23T18:06:58.294 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-06-23T18:06:58.313 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-06-23T18:06:58.332 INFO:teuthology.orchestra.run.smithi022.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-06-23T18:06:58.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:06:58.354 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec ls tests/generic) 2023-06-23T18:06:58.363 INFO:teuthology.orchestra.run.smithi022.stdout:001 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:001.out 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:002 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:002.out 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:003 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:003.out 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:004 2023-06-23T18:06:58.364 INFO:teuthology.orchestra.run.smithi022.stdout:004.out 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:005 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:005.out 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:006 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:006.out 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:007 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:007.out 2023-06-23T18:06:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:008 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:008.out 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:009 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:009.out 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:010 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:010.out 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:011 2023-06-23T18:06:58.366 INFO:teuthology.orchestra.run.smithi022.stdout:011.out 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:012 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:012.out 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:013 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:013.out 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:014 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:014.out 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:015 2023-06-23T18:06:58.367 INFO:teuthology.orchestra.run.smithi022.stdout:015.out 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:016 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:016.out 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:017 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:017.out 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:018 2023-06-23T18:06:58.368 INFO:teuthology.orchestra.run.smithi022.stdout:018.out 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:019 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:019.out 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:020 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:020.out 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:021 2023-06-23T18:06:58.369 INFO:teuthology.orchestra.run.smithi022.stdout:021.out 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:022 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:022.out 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:023 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:023.out 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:024 2023-06-23T18:06:58.370 INFO:teuthology.orchestra.run.smithi022.stdout:024.out 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:025 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:025.out 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:026 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:026.out 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:027 2023-06-23T18:06:58.371 INFO:teuthology.orchestra.run.smithi022.stdout:027.out 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:028 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:028.out 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:029 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:029.out 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:030 2023-06-23T18:06:58.372 INFO:teuthology.orchestra.run.smithi022.stdout:030.out 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:031 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:031.out 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:032 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:032.out 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:033 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:033.out 2023-06-23T18:06:58.373 INFO:teuthology.orchestra.run.smithi022.stdout:034 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:034.out 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:035 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:035.cfg 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:035.out.default 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:035.out.nfs 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:036 2023-06-23T18:06:58.374 INFO:teuthology.orchestra.run.smithi022.stdout:036.out 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:037 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:037.out 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:038 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:038.out 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:039 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:039.out 2023-06-23T18:06:58.375 INFO:teuthology.orchestra.run.smithi022.stdout:040 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:040.out 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:041 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:041.out 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:042 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:042.out 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:043 2023-06-23T18:06:58.376 INFO:teuthology.orchestra.run.smithi022.stdout:043.out 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:044 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:044.out 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:045 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:045.out 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:046 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:046.out 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:047 2023-06-23T18:06:58.377 INFO:teuthology.orchestra.run.smithi022.stdout:047.out 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:048 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:048.out 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:049 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:049.out 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:050 2023-06-23T18:06:58.378 INFO:teuthology.orchestra.run.smithi022.stdout:050.cfg 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:050.out.default 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:050.out.nojournal 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:050.out.xfsquota 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:051 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:051.out 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:052 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:052.out 2023-06-23T18:06:58.379 INFO:teuthology.orchestra.run.smithi022.stdout:053 2023-06-23T18:06:58.380 INFO:teuthology.orchestra.run.smithi022.stdout:053.out 2023-06-23T18:06:58.380 INFO:teuthology.orchestra.run.smithi022.stdout:054 2023-06-23T18:06:58.380 INFO:teuthology.orchestra.run.smithi022.stdout:054.out 2023-06-23T18:06:58.380 INFO:teuthology.orchestra.run.smithi022.stdout:055 2023-06-23T18:06:58.380 INFO:teuthology.orchestra.run.smithi022.stdout:055.out 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:056 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:056.out 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:057 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:057.out 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:058 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:058.out 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:059 2023-06-23T18:06:58.381 INFO:teuthology.orchestra.run.smithi022.stdout:059.out 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:060 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:060.out 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:061 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:061.out 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:062 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:062.out 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:063 2023-06-23T18:06:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:063.out 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:064 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:064.out 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:065 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:065.out 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:066 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:066.out 2023-06-23T18:06:58.383 INFO:teuthology.orchestra.run.smithi022.stdout:067 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:067.out 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:068 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:068.out 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:069 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:069.out 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:070 2023-06-23T18:06:58.384 INFO:teuthology.orchestra.run.smithi022.stdout:070.out 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:071 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:071.out 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:072 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:072.out 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:073 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:073.out 2023-06-23T18:06:58.385 INFO:teuthology.orchestra.run.smithi022.stdout:074 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:074.out 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:075 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:075.out 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:076 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:076.out 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:077 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:077.out 2023-06-23T18:06:58.386 INFO:teuthology.orchestra.run.smithi022.stdout:078 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:078.out 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:079 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:079.out 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:080 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:080.out 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:081 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:081.out 2023-06-23T18:06:58.387 INFO:teuthology.orchestra.run.smithi022.stdout:082 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:082.out 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:083 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:083.out 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:084 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:084.out 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:085 2023-06-23T18:06:58.388 INFO:teuthology.orchestra.run.smithi022.stdout:085.out 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:086 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:086.out 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:087 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:087.out 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:088 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:088.out 2023-06-23T18:06:58.389 INFO:teuthology.orchestra.run.smithi022.stdout:089 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:089.out 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:090 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:090.out 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:091 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:091.out 2023-06-23T18:06:58.390 INFO:teuthology.orchestra.run.smithi022.stdout:092 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:092.out 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:093 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:093.out 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:094 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:094.out 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:095 2023-06-23T18:06:58.391 INFO:teuthology.orchestra.run.smithi022.stdout:095.out 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:096 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:096.out 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:097 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:097.out 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:098 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:098.out 2023-06-23T18:06:58.392 INFO:teuthology.orchestra.run.smithi022.stdout:099 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:099.out 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:100 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:100.out 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:101 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:101.out 2023-06-23T18:06:58.393 INFO:teuthology.orchestra.run.smithi022.stdout:102 2023-06-23T18:06:58.394 INFO:teuthology.orchestra.run.smithi022.stdout:102.out 2023-06-23T18:06:58.394 INFO:teuthology.orchestra.run.smithi022.stdout:103 2023-06-23T18:06:58.394 INFO:teuthology.orchestra.run.smithi022.stdout:103.out 2023-06-23T18:06:58.394 INFO:teuthology.orchestra.run.smithi022.stdout:104 2023-06-23T18:06:58.394 INFO:teuthology.orchestra.run.smithi022.stdout:104.out 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:105 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:105.out 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:106 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:106.out 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:107 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:107.out 2023-06-23T18:06:58.395 INFO:teuthology.orchestra.run.smithi022.stdout:108 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:108.out 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:109 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:109.out 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:110 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:110.out 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:111 2023-06-23T18:06:58.396 INFO:teuthology.orchestra.run.smithi022.stdout:111.out 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:112 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:112.out 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:113 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:113.out 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:114 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:114.out 2023-06-23T18:06:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:115 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:115.out 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:116 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:116.out 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:117 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:117.out 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:118 2023-06-23T18:06:58.398 INFO:teuthology.orchestra.run.smithi022.stdout:118.out 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:119 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:119.out 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:120 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:120.out 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:121 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:121.out 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:122 2023-06-23T18:06:58.399 INFO:teuthology.orchestra.run.smithi022.stdout:122.out 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:123 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:123.out 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:124 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:124.out 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:125 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:125.out 2023-06-23T18:06:58.400 INFO:teuthology.orchestra.run.smithi022.stdout:126 2023-06-23T18:06:58.401 INFO:teuthology.orchestra.run.smithi022.stdout:126.out 2023-06-23T18:06:58.401 INFO:teuthology.orchestra.run.smithi022.stdout:127 2023-06-23T18:06:58.401 INFO:teuthology.orchestra.run.smithi022.stdout:127.out 2023-06-23T18:06:58.401 INFO:teuthology.orchestra.run.smithi022.stdout:128 2023-06-23T18:06:58.401 INFO:teuthology.orchestra.run.smithi022.stdout:128.out 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:129 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:129.out 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:130 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:130.out 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:131 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:131.out 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:132 2023-06-23T18:06:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:132.out 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:133 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:133.out 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:134 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:134.out 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:135 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:135.out 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:136 2023-06-23T18:06:58.403 INFO:teuthology.orchestra.run.smithi022.stdout:136.out 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:137 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:137.out 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:138 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:138.out 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:139 2023-06-23T18:06:58.404 INFO:teuthology.orchestra.run.smithi022.stdout:139.out 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:140 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:140.out 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:141 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:141.out 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:142 2023-06-23T18:06:58.405 INFO:teuthology.orchestra.run.smithi022.stdout:142.out 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:143 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:143.out 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:144 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:144.out 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:145 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:145.out 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:146 2023-06-23T18:06:58.406 INFO:teuthology.orchestra.run.smithi022.stdout:146.out 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:147 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:147.out 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:148 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:148.out 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:149 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:149.out 2023-06-23T18:06:58.407 INFO:teuthology.orchestra.run.smithi022.stdout:150 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:150.out 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:151 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:151.out 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:152 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:152.out 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:153 2023-06-23T18:06:58.408 INFO:teuthology.orchestra.run.smithi022.stdout:153.out 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:154 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:154.out 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:155 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:155.out 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:156 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:156.out 2023-06-23T18:06:58.409 INFO:teuthology.orchestra.run.smithi022.stdout:157 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:157.out 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:158 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:158.out 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:159 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:159.out 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:160 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:160.out 2023-06-23T18:06:58.410 INFO:teuthology.orchestra.run.smithi022.stdout:161 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:161.out 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:162 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:162.out 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:163 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:163.out 2023-06-23T18:06:58.411 INFO:teuthology.orchestra.run.smithi022.stdout:164 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:164.out 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:165 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:165.out 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:166 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:166.out 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:167 2023-06-23T18:06:58.412 INFO:teuthology.orchestra.run.smithi022.stdout:167.out 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:168 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:168.out 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:169 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:169.out 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:170 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:170.out 2023-06-23T18:06:58.413 INFO:teuthology.orchestra.run.smithi022.stdout:171 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:171.out 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:172 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:172.out 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:173 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:173.out 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:174 2023-06-23T18:06:58.414 INFO:teuthology.orchestra.run.smithi022.stdout:174.out 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:175 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:175.out 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:176 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:176.out 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:177 2023-06-23T18:06:58.415 INFO:teuthology.orchestra.run.smithi022.stdout:177.out 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:178 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:178.out 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:179 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:179.out 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:180 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:180.out 2023-06-23T18:06:58.416 INFO:teuthology.orchestra.run.smithi022.stdout:181 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:181.out 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:182 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:182.out 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:183 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:183.out 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:184 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:184.out 2023-06-23T18:06:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:185 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:185.out 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:186 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:186.out 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:187 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:187.out 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:188 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:188.out 2023-06-23T18:06:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:189 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:189.out 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:190 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:190.out 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:191 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:191.out 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:192 2023-06-23T18:06:58.419 INFO:teuthology.orchestra.run.smithi022.stdout:192.out 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:193 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:193.out 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:194 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:194.out 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:195 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:195.out 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:196 2023-06-23T18:06:58.420 INFO:teuthology.orchestra.run.smithi022.stdout:196.out 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:197 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:197.out 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:198 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:198.out 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:199 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:199.out 2023-06-23T18:06:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:200 2023-06-23T18:06:58.422 INFO:teuthology.orchestra.run.smithi022.stdout:200.out 2023-06-23T18:06:58.422 INFO:teuthology.orchestra.run.smithi022.stdout:201 2023-06-23T18:06:58.422 INFO:teuthology.orchestra.run.smithi022.stdout:201.out 2023-06-23T18:06:58.422 INFO:teuthology.orchestra.run.smithi022.stdout:202 2023-06-23T18:06:58.422 INFO:teuthology.orchestra.run.smithi022.stdout:202.out 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:203 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:203.out 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:204 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:204.out 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:205 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:205.out 2023-06-23T18:06:58.423 INFO:teuthology.orchestra.run.smithi022.stdout:206 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:206.out 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:207 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:207.out 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:208 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:208.out 2023-06-23T18:06:58.424 INFO:teuthology.orchestra.run.smithi022.stdout:209 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:209.out 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:210 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:210.out 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:211 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:211.out 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:212 2023-06-23T18:06:58.425 INFO:teuthology.orchestra.run.smithi022.stdout:212.out 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:213 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:213.out 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:214 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:214.out 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:215 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:215.out 2023-06-23T18:06:58.426 INFO:teuthology.orchestra.run.smithi022.stdout:216 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:216.out 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:217 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:217.out 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:218 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:218.out 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:219 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:219.out 2023-06-23T18:06:58.427 INFO:teuthology.orchestra.run.smithi022.stdout:220 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:220.out 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:221 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:221.out 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:222 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:222.out 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:223 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:223.out 2023-06-23T18:06:58.428 INFO:teuthology.orchestra.run.smithi022.stdout:224 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:224.out 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:225 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:225.out 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:226 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:226.out 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:227 2023-06-23T18:06:58.429 INFO:teuthology.orchestra.run.smithi022.stdout:227.out 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:228 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:228.out 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:229 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:229.out 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:230 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:230.out 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:231 2023-06-23T18:06:58.430 INFO:teuthology.orchestra.run.smithi022.stdout:231.out 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:232 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:232.out 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:233 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:233.out 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:234 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:234.out 2023-06-23T18:06:58.431 INFO:teuthology.orchestra.run.smithi022.stdout:235 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:235.out 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:236 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:236.out 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:237 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:237.out 2023-06-23T18:06:58.432 INFO:teuthology.orchestra.run.smithi022.stdout:238 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:238.out 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:239 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:239.out 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:240 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:240.out 2023-06-23T18:06:58.433 INFO:teuthology.orchestra.run.smithi022.stdout:241 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:241.out 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:242 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:242.out 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:243 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:243.out 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:244 2023-06-23T18:06:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:244.out 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:245 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:245.out 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:246 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:246.out 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:247 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:247.out 2023-06-23T18:06:58.435 INFO:teuthology.orchestra.run.smithi022.stdout:248 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:248.out 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:249 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:249.out 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:250 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:250.out 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:251 2023-06-23T18:06:58.436 INFO:teuthology.orchestra.run.smithi022.stdout:251.out 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:252 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:252.out 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:253 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:253.out 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:254 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:254.out 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:255 2023-06-23T18:06:58.437 INFO:teuthology.orchestra.run.smithi022.stdout:255.out 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:256 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:256.out 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:257 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:257.out 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:258 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:258.out 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:259 2023-06-23T18:06:58.438 INFO:teuthology.orchestra.run.smithi022.stdout:259.out 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:260 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:260.out 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:261 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:261.out 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:262 2023-06-23T18:06:58.439 INFO:teuthology.orchestra.run.smithi022.stdout:262.out 2023-06-23T18:06:58.440 INFO:teuthology.orchestra.run.smithi022.stdout:263 2023-06-23T18:06:58.440 INFO:teuthology.orchestra.run.smithi022.stdout:263.out 2023-06-23T18:06:58.440 INFO:teuthology.orchestra.run.smithi022.stdout:264 2023-06-23T18:06:58.441 INFO:teuthology.orchestra.run.smithi022.stdout:264.out 2023-06-23T18:06:58.441 INFO:teuthology.orchestra.run.smithi022.stdout:265 2023-06-23T18:06:58.441 INFO:teuthology.orchestra.run.smithi022.stdout:265.out 2023-06-23T18:06:58.441 INFO:teuthology.orchestra.run.smithi022.stdout:266 2023-06-23T18:06:58.442 INFO:teuthology.orchestra.run.smithi022.stdout:266.out 2023-06-23T18:06:58.442 INFO:teuthology.orchestra.run.smithi022.stdout:267 2023-06-23T18:06:58.442 INFO:teuthology.orchestra.run.smithi022.stdout:267.out 2023-06-23T18:06:58.442 INFO:teuthology.orchestra.run.smithi022.stdout:268 2023-06-23T18:06:58.443 INFO:teuthology.orchestra.run.smithi022.stdout:268.out 2023-06-23T18:06:58.443 INFO:teuthology.orchestra.run.smithi022.stdout:269 2023-06-23T18:06:58.443 INFO:teuthology.orchestra.run.smithi022.stdout:269.out 2023-06-23T18:06:58.444 INFO:teuthology.orchestra.run.smithi022.stdout:270 2023-06-23T18:06:58.444 INFO:teuthology.orchestra.run.smithi022.stdout:270.out 2023-06-23T18:06:58.444 INFO:teuthology.orchestra.run.smithi022.stdout:271 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:271.out 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:272 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:272.out 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:273 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:273.out 2023-06-23T18:06:58.445 INFO:teuthology.orchestra.run.smithi022.stdout:274 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:274.out 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:275 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:275.out 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:276 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:276.out 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:277 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:277.out 2023-06-23T18:06:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:278 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:278.out 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:279 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:279.out 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:280 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:280.out 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:281 2023-06-23T18:06:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:281.out 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:282 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:282.out 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:283 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:283.out 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:284 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:284.out 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:285 2023-06-23T18:06:58.448 INFO:teuthology.orchestra.run.smithi022.stdout:285.out 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:286 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:286.out 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:287 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:287.out 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:288 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:288.out 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:289 2023-06-23T18:06:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:289.out 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:290 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:290.out 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:291 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:291.out 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:292 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:292.out 2023-06-23T18:06:58.450 INFO:teuthology.orchestra.run.smithi022.stdout:293 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:293.out 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:294 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:294.out 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:295 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:295.out 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:296 2023-06-23T18:06:58.451 INFO:teuthology.orchestra.run.smithi022.stdout:296.out 2023-06-23T18:06:58.452 INFO:teuthology.orchestra.run.smithi022.stdout:297 2023-06-23T18:06:58.452 INFO:teuthology.orchestra.run.smithi022.stdout:297.out 2023-06-23T18:06:58.452 INFO:teuthology.orchestra.run.smithi022.stdout:298 2023-06-23T18:06:58.452 INFO:teuthology.orchestra.run.smithi022.stdout:298.out 2023-06-23T18:06:58.452 INFO:teuthology.orchestra.run.smithi022.stdout:299 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:299.out 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:300 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:300.out 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:301 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:301.out 2023-06-23T18:06:58.453 INFO:teuthology.orchestra.run.smithi022.stdout:302 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:302.out 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:303 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:303.out 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:304 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:304.out 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:305 2023-06-23T18:06:58.454 INFO:teuthology.orchestra.run.smithi022.stdout:305.out 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:306 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:306.out 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:307 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:307.out 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:308 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:308.out 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:309 2023-06-23T18:06:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:309.out 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:310 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:310.out 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:311 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:311.out 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:312 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:312.out 2023-06-23T18:06:58.456 INFO:teuthology.orchestra.run.smithi022.stdout:313 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:313.out 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:314 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:314.out 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:315 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:315.out 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:316 2023-06-23T18:06:58.457 INFO:teuthology.orchestra.run.smithi022.stdout:316.out 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:317 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:317.out 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:318 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:318.out 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:319 2023-06-23T18:06:58.458 INFO:teuthology.orchestra.run.smithi022.stdout:319.out 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:320 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:320.out 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:321 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:321.out 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:322 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:322.out 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:323 2023-06-23T18:06:58.459 INFO:teuthology.orchestra.run.smithi022.stdout:323.out 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:324 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:324.out 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:325 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:325.out 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:326 2023-06-23T18:06:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:326.out 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:327 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:327.out 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:328 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:328.out 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:329 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:329.out 2023-06-23T18:06:58.461 INFO:teuthology.orchestra.run.smithi022.stdout:330 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:330.out 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:331 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:331.out 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:332 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:332.out 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:333 2023-06-23T18:06:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:333.out 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:334 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:334.out 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:335 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:335.out 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:336 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:336.out 2023-06-23T18:06:58.463 INFO:teuthology.orchestra.run.smithi022.stdout:337 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:337.out 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:338 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:338.out 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:339 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:339.out 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:340 2023-06-23T18:06:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:340.out 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:341 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:341.out 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:342 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:342.out 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:343 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:343.out 2023-06-23T18:06:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:344 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:344.out 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:345 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:345.out 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:346 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:346.out 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:347 2023-06-23T18:06:58.466 INFO:teuthology.orchestra.run.smithi022.stdout:347.out 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:348 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:348.out 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:349 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:349.out 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:350 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:350.out 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:351 2023-06-23T18:06:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:351.out 2023-06-23T18:06:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:352 2023-06-23T18:06:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:352.out 2023-06-23T18:06:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:353 2023-06-23T18:06:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:353.out 2023-06-23T18:06:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:354 2023-06-23T18:06:58.469 INFO:teuthology.orchestra.run.smithi022.stdout:354.out 2023-06-23T18:06:58.469 INFO:teuthology.orchestra.run.smithi022.stdout:355 2023-06-23T18:06:58.469 INFO:teuthology.orchestra.run.smithi022.stdout:355.out 2023-06-23T18:06:58.469 INFO:teuthology.orchestra.run.smithi022.stdout:356 2023-06-23T18:06:58.469 INFO:teuthology.orchestra.run.smithi022.stdout:356.out 2023-06-23T18:06:58.470 INFO:teuthology.orchestra.run.smithi022.stdout:357 2023-06-23T18:06:58.470 INFO:teuthology.orchestra.run.smithi022.stdout:357.out 2023-06-23T18:06:58.470 INFO:teuthology.orchestra.run.smithi022.stdout:358 2023-06-23T18:06:58.470 INFO:teuthology.orchestra.run.smithi022.stdout:358.out 2023-06-23T18:06:58.471 INFO:teuthology.orchestra.run.smithi022.stdout:359 2023-06-23T18:06:58.471 INFO:teuthology.orchestra.run.smithi022.stdout:359.out 2023-06-23T18:06:58.471 INFO:teuthology.orchestra.run.smithi022.stdout:360 2023-06-23T18:06:58.472 INFO:teuthology.orchestra.run.smithi022.stdout:360.out 2023-06-23T18:06:58.472 INFO:teuthology.orchestra.run.smithi022.stdout:361 2023-06-23T18:06:58.472 INFO:teuthology.orchestra.run.smithi022.stdout:361.out 2023-06-23T18:06:58.473 INFO:teuthology.orchestra.run.smithi022.stdout:362 2023-06-23T18:06:58.473 INFO:teuthology.orchestra.run.smithi022.stdout:362.out 2023-06-23T18:06:58.473 INFO:teuthology.orchestra.run.smithi022.stdout:363 2023-06-23T18:06:58.473 INFO:teuthology.orchestra.run.smithi022.stdout:363.out 2023-06-23T18:06:58.474 INFO:teuthology.orchestra.run.smithi022.stdout:364 2023-06-23T18:06:58.474 INFO:teuthology.orchestra.run.smithi022.stdout:364.out 2023-06-23T18:06:58.474 INFO:teuthology.orchestra.run.smithi022.stdout:365 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:365.out 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:366 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:366.out 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:367 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:367.out 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:368 2023-06-23T18:06:58.475 INFO:teuthology.orchestra.run.smithi022.stdout:368.out 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:369 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:369.out 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:370 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:370.out 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:371 2023-06-23T18:06:58.476 INFO:teuthology.orchestra.run.smithi022.stdout:371.out 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:372 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:372.out 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:373 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:373.out 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:374 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:374.out 2023-06-23T18:06:58.477 INFO:teuthology.orchestra.run.smithi022.stdout:375 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:375.out 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:376 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:376.out 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:377 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:377.out 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:378 2023-06-23T18:06:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:378.out 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:379 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:379.out 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:380 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:380.out 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:381 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:381.out 2023-06-23T18:06:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:382 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:382.out 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:383 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:383.out 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:384 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:384.out 2023-06-23T18:06:58.480 INFO:teuthology.orchestra.run.smithi022.stdout:385 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:385.out 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:386 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:386.out 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:387 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:387.out 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:388 2023-06-23T18:06:58.481 INFO:teuthology.orchestra.run.smithi022.stdout:388.out 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:389 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:389.out 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:390 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:390.out 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:391 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:391.out 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:392 2023-06-23T18:06:58.482 INFO:teuthology.orchestra.run.smithi022.stdout:392.out 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:393 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:393.out 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:394 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:394.out 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:395 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:395.out 2023-06-23T18:06:58.483 INFO:teuthology.orchestra.run.smithi022.stdout:396 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:396.out 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:397 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:397.out 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:398 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:398.out 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:399 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:399.out 2023-06-23T18:06:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:400 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:400.out 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:401 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:401.out 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:402 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:402.out 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:403 2023-06-23T18:06:58.485 INFO:teuthology.orchestra.run.smithi022.stdout:403.out 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:404 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:404.out 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:405 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:405.out 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:406 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:406.out 2023-06-23T18:06:58.486 INFO:teuthology.orchestra.run.smithi022.stdout:407 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:407.out 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:408 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:408.out 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:409 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:409.out 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:410 2023-06-23T18:06:58.487 INFO:teuthology.orchestra.run.smithi022.stdout:410.out 2023-06-23T18:06:58.488 INFO:teuthology.orchestra.run.smithi022.stdout:411 2023-06-23T18:06:58.488 INFO:teuthology.orchestra.run.smithi022.stdout:411.out 2023-06-23T18:06:58.488 INFO:teuthology.orchestra.run.smithi022.stdout:412 2023-06-23T18:06:58.489 INFO:teuthology.orchestra.run.smithi022.stdout:412.out 2023-06-23T18:06:58.489 INFO:teuthology.orchestra.run.smithi022.stdout:413 2023-06-23T18:06:58.489 INFO:teuthology.orchestra.run.smithi022.stdout:413.out 2023-06-23T18:06:58.489 INFO:teuthology.orchestra.run.smithi022.stdout:414 2023-06-23T18:06:58.490 INFO:teuthology.orchestra.run.smithi022.stdout:414.out 2023-06-23T18:06:58.490 INFO:teuthology.orchestra.run.smithi022.stdout:415 2023-06-23T18:06:58.490 INFO:teuthology.orchestra.run.smithi022.stdout:415.out 2023-06-23T18:06:58.490 INFO:teuthology.orchestra.run.smithi022.stdout:416 2023-06-23T18:06:58.491 INFO:teuthology.orchestra.run.smithi022.stdout:416.out 2023-06-23T18:06:58.491 INFO:teuthology.orchestra.run.smithi022.stdout:417 2023-06-23T18:06:58.491 INFO:teuthology.orchestra.run.smithi022.stdout:417.out 2023-06-23T18:06:58.491 INFO:teuthology.orchestra.run.smithi022.stdout:418 2023-06-23T18:06:58.492 INFO:teuthology.orchestra.run.smithi022.stdout:418.out 2023-06-23T18:06:58.492 INFO:teuthology.orchestra.run.smithi022.stdout:419 2023-06-23T18:06:58.493 INFO:teuthology.orchestra.run.smithi022.stdout:419.out 2023-06-23T18:06:58.493 INFO:teuthology.orchestra.run.smithi022.stdout:420 2023-06-23T18:06:58.493 INFO:teuthology.orchestra.run.smithi022.stdout:420.out 2023-06-23T18:06:58.493 INFO:teuthology.orchestra.run.smithi022.stdout:421 2023-06-23T18:06:58.494 INFO:teuthology.orchestra.run.smithi022.stdout:421.out 2023-06-23T18:06:58.494 INFO:teuthology.orchestra.run.smithi022.stdout:422 2023-06-23T18:06:58.494 INFO:teuthology.orchestra.run.smithi022.stdout:422.out 2023-06-23T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stdout:423 2023-06-23T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stdout:423.out 2023-06-23T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stdout:424 2023-06-23T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stdout:424.out 2023-06-23T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stdout:425 2023-06-23T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stdout:425.out 2023-06-23T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stdout:426 2023-06-23T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stdout:426.out 2023-06-23T18:06:58.497 INFO:teuthology.orchestra.run.smithi022.stdout:427 2023-06-23T18:06:58.497 INFO:teuthology.orchestra.run.smithi022.stdout:427.out 2023-06-23T18:06:58.497 INFO:teuthology.orchestra.run.smithi022.stdout:428 2023-06-23T18:06:58.497 INFO:teuthology.orchestra.run.smithi022.stdout:428.out 2023-06-23T18:06:58.498 INFO:teuthology.orchestra.run.smithi022.stdout:429 2023-06-23T18:06:58.498 INFO:teuthology.orchestra.run.smithi022.stdout:429.out 2023-06-23T18:06:58.498 INFO:teuthology.orchestra.run.smithi022.stdout:430 2023-06-23T18:06:58.498 INFO:teuthology.orchestra.run.smithi022.stdout:430.out 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:431 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:431.out 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:432 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:432.out 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:433 2023-06-23T18:06:58.499 INFO:teuthology.orchestra.run.smithi022.stdout:433.out 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:434 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:434.out 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:435 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:435.out 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:436 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:436.out 2023-06-23T18:06:58.500 INFO:teuthology.orchestra.run.smithi022.stdout:437 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:437.out 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:438 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:438.out 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:439 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:439.out 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:440 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:440.out 2023-06-23T18:06:58.501 INFO:teuthology.orchestra.run.smithi022.stdout:441 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:441.out 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:442 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:442.out 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:443 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:443.out 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:444 2023-06-23T18:06:58.502 INFO:teuthology.orchestra.run.smithi022.stdout:444.out 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:445 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:445.out 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:446 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:446.out 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:447 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:447.out 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:448 2023-06-23T18:06:58.503 INFO:teuthology.orchestra.run.smithi022.stdout:448.out 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:449 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:449.out 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:450 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:450.out 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:451 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:451.out 2023-06-23T18:06:58.504 INFO:teuthology.orchestra.run.smithi022.stdout:452 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:452.out 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:453 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:453.out 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:454 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:454.out 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:455 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:455.out 2023-06-23T18:06:58.505 INFO:teuthology.orchestra.run.smithi022.stdout:456 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:456.out 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:457 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:457.out 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:458 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:458.out 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:459 2023-06-23T18:06:58.506 INFO:teuthology.orchestra.run.smithi022.stdout:459.out 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:460 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:460.out 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:461 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:461.out 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:462 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:462.out 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:463 2023-06-23T18:06:58.507 INFO:teuthology.orchestra.run.smithi022.stdout:463.out 2023-06-23T18:06:58.508 INFO:teuthology.orchestra.run.smithi022.stdout:464 2023-06-23T18:06:58.508 INFO:teuthology.orchestra.run.smithi022.stdout:464.out 2023-06-23T18:06:58.508 INFO:teuthology.orchestra.run.smithi022.stdout:465 2023-06-23T18:06:58.508 INFO:teuthology.orchestra.run.smithi022.stdout:465.out 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:466 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:466.out 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:467 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:467.out 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:468 2023-06-23T18:06:58.509 INFO:teuthology.orchestra.run.smithi022.stdout:468.out 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:469 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:469.out 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:470 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:470.out 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:471 2023-06-23T18:06:58.510 INFO:teuthology.orchestra.run.smithi022.stdout:471.out 2023-06-23T18:06:58.511 INFO:teuthology.orchestra.run.smithi022.stdout:472 2023-06-23T18:06:58.511 INFO:teuthology.orchestra.run.smithi022.stdout:472.out 2023-06-23T18:06:58.511 INFO:teuthology.orchestra.run.smithi022.stdout:473 2023-06-23T18:06:58.511 INFO:teuthology.orchestra.run.smithi022.stdout:473.out 2023-06-23T18:06:58.512 INFO:teuthology.orchestra.run.smithi022.stdout:474 2023-06-23T18:06:58.512 INFO:teuthology.orchestra.run.smithi022.stdout:474.out 2023-06-23T18:06:58.512 INFO:teuthology.orchestra.run.smithi022.stdout:475 2023-06-23T18:06:58.512 INFO:teuthology.orchestra.run.smithi022.stdout:475.out 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:476 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:476.out 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:477 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:477.out 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:478 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:478.out 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:479 2023-06-23T18:06:58.513 INFO:teuthology.orchestra.run.smithi022.stdout:479.out 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:480 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:480.out 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:481 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:481.out 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:482 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:482.out 2023-06-23T18:06:58.514 INFO:teuthology.orchestra.run.smithi022.stdout:483 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:483.out 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:484 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:484.out 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:485 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:485.out 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:486 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:486.out 2023-06-23T18:06:58.515 INFO:teuthology.orchestra.run.smithi022.stdout:487 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:487.out 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:488 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:488.out 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:489 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:489.out 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:490 2023-06-23T18:06:58.516 INFO:teuthology.orchestra.run.smithi022.stdout:490.out 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:491 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:491.out 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:492 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:492.out 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:493 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:493.out 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:494 2023-06-23T18:06:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:494.out 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:495 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:495.out 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:496 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:496.out 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:497 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:497.out 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:498 2023-06-23T18:06:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:498.out 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:499 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:499.out 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:500 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:500.out 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:501 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:501.out 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:502 2023-06-23T18:06:58.519 INFO:teuthology.orchestra.run.smithi022.stdout:502.out 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:503 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:503.out 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:504 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:504.out 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:505 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:505.out 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:506 2023-06-23T18:06:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:506.out 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:507 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:507.out 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:508 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:508.out 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:509 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:509.out 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:510 2023-06-23T18:06:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:510.out 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:511 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:511.out 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:512 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:512.out 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:513 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:513.out 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:514 2023-06-23T18:06:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:514.out 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:515 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:515.out 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:516 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:516.out 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:517 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:517.out 2023-06-23T18:06:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:518 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:518.out 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:519 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:519.out 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:520 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:520.out 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:521 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:521.out 2023-06-23T18:06:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:522 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:522.out 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:523 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:523.out 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:524 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:524.out 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:525 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:525.out 2023-06-23T18:06:58.525 INFO:teuthology.orchestra.run.smithi022.stdout:526 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:526.out 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:527 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:527.out 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:528 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:528.out 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:529 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:529.out 2023-06-23T18:06:58.526 INFO:teuthology.orchestra.run.smithi022.stdout:530 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:530.out 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:531 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:531.out 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:532 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:532.out 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:533 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:533.out 2023-06-23T18:06:58.527 INFO:teuthology.orchestra.run.smithi022.stdout:534 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:534.out 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:535 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:535.out 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:536 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:536.out 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:537 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:537.out 2023-06-23T18:06:58.528 INFO:teuthology.orchestra.run.smithi022.stdout:538 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:538.out 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:539 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:539.out 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:540 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:540.out 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:541 2023-06-23T18:06:58.529 INFO:teuthology.orchestra.run.smithi022.stdout:541.out 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:542 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:542.out 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:543 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:543.out 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:544 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:544.out 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:545 2023-06-23T18:06:58.530 INFO:teuthology.orchestra.run.smithi022.stdout:545.out 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:546 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:546.out 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:547 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:547.out 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:548 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:548.out 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:549 2023-06-23T18:06:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:549.out 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:550 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:550.out 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:551 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:551.out 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:552 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:552.out 2023-06-23T18:06:58.532 INFO:teuthology.orchestra.run.smithi022.stdout:553 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:553.out 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:554 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:554.out 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:555 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:555.out 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:556 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:556.out 2023-06-23T18:06:58.533 INFO:teuthology.orchestra.run.smithi022.stdout:557 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:557.out 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:558 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:558.out 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:559 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:559.out 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:560 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:560.out 2023-06-23T18:06:58.534 INFO:teuthology.orchestra.run.smithi022.stdout:561 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:561.out 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:562 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:562.out 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:563 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:563.out 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:564 2023-06-23T18:06:58.535 INFO:teuthology.orchestra.run.smithi022.stdout:564.out 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:565 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:565.out 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:566 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:566.out 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:567 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:567.out 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:568 2023-06-23T18:06:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:568.out 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:569 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:569.out 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:570 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:570.out 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:571 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:571.out 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:572 2023-06-23T18:06:58.537 INFO:teuthology.orchestra.run.smithi022.stdout:572.out 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:573 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:573.out 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:574 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:574.out 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:575 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:575.out 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:576 2023-06-23T18:06:58.538 INFO:teuthology.orchestra.run.smithi022.stdout:576.out 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:577 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:577.out 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:578 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:578.out 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:579 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:579.out 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:580 2023-06-23T18:06:58.539 INFO:teuthology.orchestra.run.smithi022.stdout:580.out 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:581 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:581.out 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:582 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:582.out 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:583 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:583.out 2023-06-23T18:06:58.540 INFO:teuthology.orchestra.run.smithi022.stdout:584 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:584.out 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:585 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:585.out 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:586 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:586.out 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:587 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:587.out 2023-06-23T18:06:58.541 INFO:teuthology.orchestra.run.smithi022.stdout:588 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:588.out 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:589 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:589.out 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:590 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:590.out 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:591 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:591.out 2023-06-23T18:06:58.542 INFO:teuthology.orchestra.run.smithi022.stdout:592 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:592.out 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:593 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:593.out 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:594 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:594.out 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:595 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:595.out 2023-06-23T18:06:58.543 INFO:teuthology.orchestra.run.smithi022.stdout:596 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:596.out 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:597 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:597.out 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:598 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:598.out 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:599 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:599.out 2023-06-23T18:06:58.544 INFO:teuthology.orchestra.run.smithi022.stdout:600 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:600.out 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:601 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:601.out 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:602 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:602.out 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:603 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:603.out 2023-06-23T18:06:58.545 INFO:teuthology.orchestra.run.smithi022.stdout:604 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:604.out 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:605 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:605.out 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:606 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:606.out 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:607 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:607.out 2023-06-23T18:06:58.546 INFO:teuthology.orchestra.run.smithi022.stdout:608 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:608.out 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:609 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:609.out 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:610 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:610.out 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:611 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:611.out 2023-06-23T18:06:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:612 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:612.out 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:613 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:613.out 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:614 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:614.out 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:615 2023-06-23T18:06:58.548 INFO:teuthology.orchestra.run.smithi022.stdout:615.out 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:616 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:616.out 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:617 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:617.out 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:618 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:618.out 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:619 2023-06-23T18:06:58.549 INFO:teuthology.orchestra.run.smithi022.stdout:619.out 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:620 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:620.out 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:621 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:621.out 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:622 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:622.out 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:623 2023-06-23T18:06:58.550 INFO:teuthology.orchestra.run.smithi022.stdout:623.out 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:624 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:624.out 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:625 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:625.out 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:626 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:626.out 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:627 2023-06-23T18:06:58.551 INFO:teuthology.orchestra.run.smithi022.stdout:627.out 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:628 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:628.out 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:629 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:629.out 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:630 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:630.out 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:631 2023-06-23T18:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:631.out 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:632 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:632.out 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:633 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:633.out 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:634 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:634.out 2023-06-23T18:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout:635 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:635.out 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:636 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:636.out 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:637 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:637.out 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:638 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:638.out 2023-06-23T18:06:58.554 INFO:teuthology.orchestra.run.smithi022.stdout:639 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:639.out 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:640 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:640.out 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:641 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:641.out 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:642 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:642.out 2023-06-23T18:06:58.555 INFO:teuthology.orchestra.run.smithi022.stdout:643 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:643.out 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:644 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:644.out 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:645 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:645.out 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:646 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:646.out 2023-06-23T18:06:58.556 INFO:teuthology.orchestra.run.smithi022.stdout:647 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:647.out 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:648 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:648.out 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:649 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:649.out 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:650 2023-06-23T18:06:58.557 INFO:teuthology.orchestra.run.smithi022.stdout:650.out 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:651 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:651.out 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:652 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:652.out 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:653 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:653.out 2023-06-23T18:06:58.558 INFO:teuthology.orchestra.run.smithi022.stdout:654 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:654.out 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:655 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:655.out 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:656 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:656.out 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:657 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:657.out 2023-06-23T18:06:58.559 INFO:teuthology.orchestra.run.smithi022.stdout:658 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:658.out 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:659 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:659.out 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:660 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:660.out 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:661 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:661.out 2023-06-23T18:06:58.560 INFO:teuthology.orchestra.run.smithi022.stdout:662 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:662.out 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:663 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:663.out 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:664 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:664.out 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:665 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:665.out 2023-06-23T18:06:58.561 INFO:teuthology.orchestra.run.smithi022.stdout:666 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:666.out 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:667 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:667.out 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:668 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:668.out 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:669 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:669.out 2023-06-23T18:06:58.562 INFO:teuthology.orchestra.run.smithi022.stdout:670 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:670.out 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:671 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:671.out 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:672 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:672.out 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:673 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:673.out 2023-06-23T18:06:58.563 INFO:teuthology.orchestra.run.smithi022.stdout:674 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:674.out 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:675 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:675.out 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:676 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:676.out 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:677 2023-06-23T18:06:58.564 INFO:teuthology.orchestra.run.smithi022.stdout:677.out 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:678 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:678.out 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:679 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:679.out 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:680 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:680.out 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:681 2023-06-23T18:06:58.565 INFO:teuthology.orchestra.run.smithi022.stdout:681.out 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:682 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:682.out 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:683 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:683.out 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:684 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:684.out 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:685 2023-06-23T18:06:58.566 INFO:teuthology.orchestra.run.smithi022.stdout:685.out 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:686 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:686.out 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:687 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:687.out 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:688 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:688.out 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:689 2023-06-23T18:06:58.567 INFO:teuthology.orchestra.run.smithi022.stdout:689.out 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:690 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:690.out 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:691 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:691.out 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:692 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:692.out 2023-06-23T18:06:58.568 INFO:teuthology.orchestra.run.smithi022.stdout:693 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:693.out 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:694 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:694.out 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:695 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:695.out 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:696 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:696.out 2023-06-23T18:06:58.569 INFO:teuthology.orchestra.run.smithi022.stdout:697 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:697.out 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:698 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:698.out 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:699 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:699.out 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:700 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:700.out 2023-06-23T18:06:58.570 INFO:teuthology.orchestra.run.smithi022.stdout:701 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:701.out 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:702 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:702.out 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:703 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:703.out 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:704 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:704.out 2023-06-23T18:06:58.571 INFO:teuthology.orchestra.run.smithi022.stdout:705 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:705.out 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:706 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:706.out 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:707 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:707.out 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:708 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:708.out 2023-06-23T18:06:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:709 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:709.out 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:710 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:710.out 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:711 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:711.out 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:712 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:712.out 2023-06-23T18:06:58.573 INFO:teuthology.orchestra.run.smithi022.stdout:713 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:713.out 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:714 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:714.out 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:715 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:715.out 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:716 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:716.out 2023-06-23T18:06:58.574 INFO:teuthology.orchestra.run.smithi022.stdout:717 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:717.out 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:718 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:718.out 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:719 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:719.out 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:720 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:720.out 2023-06-23T18:06:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:721 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:721.out 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:722 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:722.out 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:723 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:723.out 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:724 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:724.out 2023-06-23T18:06:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:725 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:725.out 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:726 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:726.out 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:727 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:727.out 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:728 2023-06-23T18:06:58.577 INFO:teuthology.orchestra.run.smithi022.stdout:728.out 2023-06-23T18:06:58.578 INFO:teuthology.orchestra.run.smithi022.stdout:group.list 2023-06-23T18:06:58.578 INFO:teuthology.orchestra.run.smithi022.stdout:Makefile 2023-06-23T18:06:58.579 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-06-23T18:06:58.865 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:06:58.874 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:06:58.875 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:03.348 INFO:teuthology.orchestra.run.smithi022.stdout:generic/001 5s 2023-06-23T18:07:03.361 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/001 2023-06-23T18:07:03.362 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:03.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[47062]: fuse finished with error 0 and tester_r 0 2023-06-23T18:07:03.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:03.382 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-06-23T18:07:03.700 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:03.709 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:03.710 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:04.654 INFO:teuthology.orchestra.run.smithi022.stdout:generic/002 1s 2023-06-23T18:07:04.668 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/002 2023-06-23T18:07:04.668 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:04.673 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:04.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:04.691 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-06-23T18:07:05.019 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:05.029 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:05.029 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:05.278 INFO:teuthology.orchestra.run.smithi022.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2023-06-23T18:07:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/003 2023-06-23T18:07:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/003 2023-06-23T18:07:05.286 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:05.292 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:05.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:05.311 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-06-23T18:07:05.641 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:05.651 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:05.651 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:05.923 INFO:teuthology.orchestra.run.smithi022.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-06-23T18:07:05.932 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/004 2023-06-23T18:07:05.932 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/004 2023-06-23T18:07:05.932 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:05.938 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:05.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:05.956 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-06-23T18:07:06.268 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:06.278 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:07.403 INFO:teuthology.orchestra.run.smithi022.stdout:generic/005 1s 2023-06-23T18:07:07.415 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/005 2023-06-23T18:07:07.415 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:07.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:07.441 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-06-23T18:07:07.733 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:07.743 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:19.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:07:19.619 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:07:49.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:07:49.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:07:58.984 INFO:teuthology.orchestra.run.smithi022.stdout:generic/006 51s 2023-06-23T18:07:58.995 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/006 2023-06-23T18:07:58.996 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:07:59.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:07:59.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:07:59.015 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-06-23T18:07:59.335 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:07:59.345 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:07:59.345 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:08:19.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:08:19.931 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:08:50.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:08:50.302 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:09:20.597 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:09:20.599 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:09:50.027 INFO:teuthology.orchestra.run.smithi022.stdout:generic/007 111s 2023-06-23T18:09:50.038 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/007 2023-06-23T18:09:50.039 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:09:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:50.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:09:50.061 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-06-23T18:09:50.373 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:09:50.382 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:09:50.383 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:50.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:09:50.697 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:09:52.573 INFO:teuthology.orchestra.run.smithi022.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-23T18:09:52.581 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/008 2023-06-23T18:09:52.581 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/008 2023-06-23T18:09:52.581 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:09:52.587 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:52.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:09:52.605 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-06-23T18:09:52.928 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:09:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:09:52.938 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:54.794 INFO:teuthology.orchestra.run.smithi022.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-23T18:09:54.803 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/009 2023-06-23T18:09:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/009 2023-06-23T18:09:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:09:54.810 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:54.828 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:09:54.828 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-06-23T18:09:55.132 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:09:55.141 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:09:55.142 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:55.411 INFO:teuthology.orchestra.run.smithi022.stdout:generic/010 [not run] /tmp/tmp.420M2hxqAdxfstests-dev/src/dbtest not built 2023-06-23T18:09:55.420 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/010 2023-06-23T18:09:55.420 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/010 2023-06-23T18:09:55.420 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:09:55.426 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:09:55.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:09:55.443 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-06-23T18:09:55.771 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:09:55.781 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:09:55.782 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:10:20.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:10:20.786 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:10:48.746 INFO:teuthology.orchestra.run.smithi022.stdout:generic/011 53s 2023-06-23T18:10:48.759 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/011 2023-06-23T18:10:48.759 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:10:48.765 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:10:48.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:10:48.786 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-06-23T18:10:49.121 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:10:49.130 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:10:49.130 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:10:50.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:10:50.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:10:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-23T18:10:51.205 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/012 2023-06-23T18:10:51.206 INFO:teuthology.orchestra.run.smithi022.stdout:Not run: generic/012 2023-06-23T18:10:51.206 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:10:51.211 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:10:51.229 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:10:51.230 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-06-23T18:10:51.550 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:10:51.558 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:10:51.559 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:11:21.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:11:21.177 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:11:51.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:11:51.505 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:12:21.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:12:21.725 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:12:26.956 INFO:tasks.ceph.mds.c.smithi022.stderr:/build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f9660b1f700 time 2023-06-23T18:12:26.948242+0000 2023-06-23T18:12:26.957 INFO:tasks.ceph.mds.c.smithi022.stderr:/build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-23T18:12:26.958 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7f966cf67257] 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.959 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.960 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: clone() 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr:*** Caught signal (Aborted) ** 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr: in thread 7f9660b1f700 thread_name:MR_Finisher 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr:2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 /build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f9660b1f700 time 2023-06-23T18:12:26.948242+0000 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr:/build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.961 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7f966cf67257] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.962 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.963 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: clone() 2023-06-23T18:12:26.964 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.964 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.964 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f966cbff420] 2023-06-23T18:12:26.964 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: gsignal() 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: abort() 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x182) [0x7f966cf672b1] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.965 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.966 INFO:tasks.ceph.mds.c.smithi022.stderr: 18: clone() 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr:2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 *** Caught signal (Aborted) ** 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: in thread 7f9660b1f700 thread_name:MR_Finisher 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f966cbff420] 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: gsignal() 2023-06-23T18:12:26.967 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: abort() 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x182) [0x7f966cf672b1] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.968 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.969 INFO:tasks.ceph.mds.c.smithi022.stderr: 18: clone() 2023-06-23T18:12:26.970 INFO:tasks.ceph.mds.c.smithi022.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-23T18:12:26.970 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.981 INFO:tasks.ceph.mds.c.smithi022.stderr: -4> 2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 /build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f9660b1f700 time 2023-06-23T18:12:26.948242+0000 2023-06-23T18:12:26.981 INFO:tasks.ceph.mds.c.smithi022.stderr:/build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-23T18:12:26.981 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.981 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.981 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7f966cf67257] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.982 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: clone() 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.983 INFO:tasks.ceph.mds.c.smithi022.stderr: 0> 2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 *** Caught signal (Aborted) ** 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: in thread 7f9660b1f700 thread_name:MR_Finisher 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f966cbff420] 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: gsignal() 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: abort() 2023-06-23T18:12:26.984 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x182) [0x7f966cf672b1] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.985 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: 18: clone() 2023-06-23T18:12:26.986 INFO:tasks.ceph.mds.c.smithi022.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-23T18:12:26.987 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.991 INFO:tasks.ceph.mds.c.smithi022.stderr: -26> 2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 /build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f9660b1f700 time 2023-06-23T18:12:26.948242+0000 2023-06-23T18:12:26.991 INFO:tasks.ceph.mds.c.smithi022.stderr:/build/ceph-18.0.0-4479-g8627af3e/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7f966cf67257] 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.992 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.993 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: clone() 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: -22> 2023-06-23T18:12:26.946+0000 7f9660b1f700 -1 *** Caught signal (Aborted) ** 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: in thread 7f9660b1f700 thread_name:MR_Finisher 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:12:26.994 INFO:tasks.ceph.mds.c.smithi022.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f966cbff420] 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 2: gsignal() 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 3: abort() 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x182) [0x7f966cf672b1] 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 5: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7f966cf67413] 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x558dbd5061c2] 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x558dbd4efa3c] 2023-06-23T18:12:26.995 INFO:tasks.ceph.mds.c.smithi022.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x3f0) [0x558dbd4f02a0] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x558dbd4f0cff] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x48d) [0x558dbd35bc7d] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x558dbd35cac0] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2e1) [0x558dbd35f321] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 13: (MDSContext::complete(int)+0x5f) [0x558dbd6309cf] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 14: (MDSIOContextBase::complete(int)+0x514) [0x558dbd631134] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x558dbd6314cb] 2023-06-23T18:12:26.996 INFO:tasks.ceph.mds.c.smithi022.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f966d00dd9d] 2023-06-23T18:12:26.997 INFO:tasks.ceph.mds.c.smithi022.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f966cbf3609] 2023-06-23T18:12:26.997 INFO:tasks.ceph.mds.c.smithi022.stderr: 18: clone() 2023-06-23T18:12:26.997 INFO:tasks.ceph.mds.c.smithi022.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-23T18:12:26.997 INFO:tasks.ceph.mds.c.smithi022.stderr: 2023-06-23T18:12:28.051 INFO:tasks.ceph.mds.c.smithi022.stderr:daemon-helper: command crashed with signal 6 2023-06-23T18:12:34.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-06-23T18:12:40.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-06-23T18:12:47.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-06-23T18:12:51.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:12:51.790 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:12:52.699 INFO:tasks.ceph.mds.b.smithi169.stderr:2023-06-23T18:12:52.691+0000 7faddacf2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:12:53.883 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-06-23T18:13:00.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~26s 2023-06-23T18:13:07.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~33s 2023-06-23T18:13:10.965 INFO:teuthology.orchestra.run.smithi022.stdout:generic/013 139s 2023-06-23T18:13:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:Ran: generic/013 2023-06-23T18:13:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:Passed all 1 tests 2023-06-23T18:13:10.978 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:13:10.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-23T18:13:10.993 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/tmp.420M2hxqAdxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-06-23T18:13:11.318 INFO:teuthology.orchestra.run.smithi022.stdout:FSTYP -- ceph-fuse 2023-06-23T18:13:11.325 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM -- Linux/x86_64 smithi022 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 2023-06-23T18:13:11.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-06-23T18:13:13.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~40s 2023-06-23T18:13:20.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~46s 2023-06-23T18:13:22.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:13:22.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:13:26.943 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~53s 2023-06-23T18:13:33.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~60s 2023-06-23T18:13:40.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~66s 2023-06-23T18:13:46.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~73s 2023-06-23T18:13:52.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:13:52.607 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:13:53.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~79s 2023-06-23T18:14:00.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~86s 2023-06-23T18:14:06.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~93s 2023-06-23T18:14:13.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~99s 2023-06-23T18:14:19.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~106s 2023-06-23T18:14:22.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:14:22.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:14:26.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~112s 2023-06-23T18:14:33.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~119s 2023-06-23T18:14:39.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~126s 2023-06-23T18:14:46.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~132s 2023-06-23T18:14:52.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~139s 2023-06-23T18:14:52.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:14:52.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:14:59.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~145s 2023-06-23T18:15:06.116 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~152s 2023-06-23T18:15:12.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~159s 2023-06-23T18:15:19.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~165s 2023-06-23T18:15:23.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:15:23.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:15:25.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~172s 2023-06-23T18:15:32.562 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~179s 2023-06-23T18:15:39.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~185s 2023-06-23T18:15:45.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~192s 2023-06-23T18:15:52.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~198s 2023-06-23T18:15:53.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:15:53.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:15:59.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~205s 2023-06-23T18:16:05.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~212s 2023-06-23T18:16:12.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~218s 2023-06-23T18:16:18.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~225s 2023-06-23T18:16:23.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:16:23.575 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:16:25.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~231s 2023-06-23T18:16:32.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~238s 2023-06-23T18:16:38.688 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~245s 2023-06-23T18:16:45.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~251s 2023-06-23T18:16:51.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~258s 2023-06-23T18:16:53.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:16:53.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:16:58.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~264s 2023-06-23T18:17:05.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~271s 2023-06-23T18:17:11.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~278s 2023-06-23T18:17:18.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~284s 2023-06-23T18:17:23.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:17:23.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:17:25.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~291s 2023-06-23T18:17:31.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~298s 2023-06-23T18:17:38.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~304s 2023-06-23T18:17:38.239 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-06-23T18:17:38.239 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:17:38.239 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:17:38.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349460k,nr_inodes=4087365,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276100k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276096k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276100k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 ceph-fuse /tmp/tmp.qA86lK7wK3test fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-06-23T18:17:38.263 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi022.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-06-23T18:17:38.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:17:38.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:17:38.264 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-23T18:17:38.344 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-06-23T18:17:38.344 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:17:38.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349460k,nr_inodes=4087365,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276100k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276096k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276100k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 ceph-fuse /tmp/tmp.qA86lK7wK3test fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-06-23T18:17:38.352 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi022.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-06-23T18:17:38.352 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-06-23T18:17:38.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:17:38.352 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-23T18:17:38.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:17:38.362 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-06-23T18:17:38.363 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:17:38.363 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:17:38.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346376k,nr_inodes=4086594,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275484k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18908 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275480k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275484k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-06-23T18:17:38.369 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi169.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-06-23T18:17:38.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:17:38.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:17:38.369 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-23T18:17:38.375 INFO:teuthology.orchestra.run.smithi169.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-06-23T18:17:38.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:17:38.377 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-06-23T18:17:38.378 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:17:38.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346376k,nr_inodes=4086594,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275484k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18908 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275480k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275484k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-06-23T18:17:38.384 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi169.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-06-23T18:17:38.385 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-06-23T18:17:38.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:17:38.385 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-23T18:17:38.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:17:38.442 INFO:teuthology.orchestra.run.smithi169.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-06-23T18:17:40.052 INFO:tasks.ceph.osd.0:Sent signal 15 2023-06-23T18:17:40.053 INFO:tasks.ceph.osd.1:Sent signal 15 2023-06-23T18:17:40.054 INFO:tasks.ceph.osd.2:Sent signal 15 2023-06-23T18:17:40.054 INFO:tasks.ceph.osd.3:Sent signal 15 2023-06-23T18:17:40.055 INFO:tasks.ceph.osd.4:Sent signal 15 2023-06-23T18:17:40.055 INFO:tasks.ceph.osd.5:Sent signal 15 2023-06-23T18:17:40.056 INFO:tasks.ceph.osd.6:Sent signal 15 2023-06-23T18:17:40.056 INFO:tasks.ceph.osd.7:Sent signal 15 2023-06-23T18:17:40.057 INFO:tasks.ceph.mds.a:Sent signal 15 2023-06-23T18:17:40.058 INFO:tasks.ceph.mds.b:Sent signal 15 2023-06-23T18:17:40.058 INFO:tasks.ceph.mds.d:Sent signal 15 2023-06-23T18:17:40.059 INFO:tasks.ceph.mon.a:Sent signal 15 2023-06-23T18:17:40.059 INFO:tasks.ceph.mon.b:Sent signal 15 2023-06-23T18:17:40.060 INFO:tasks.ceph.mon.c:Sent signal 15 2023-06-23T18:17:40.061 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-06-23T18:17:40.061 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-06-23T18:17:40.062 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:17:40.048+0000 7f776d426700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35456) UID: 0 2023-06-23T18:17:40.062 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:17:40.048+0000 7f776d426700 -1 osd.4 42 *** Got signal Terminated *** 2023-06-23T18:17:40.063 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-06-23T18:17:40.048+0000 7f776d426700 -1 osd.4 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.063 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fb0f9e83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 35463) UID: 0 2023-06-23T18:17:40.063 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fb0f9e83700 -1 osd.6 42 *** Got signal Terminated *** 2023-06-23T18:17:40.063 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fb0f9e83700 -1 osd.6 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.064 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fa2f24aa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35461) UID: 0 2023-06-23T18:17:40.064 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fa2f24aa700 -1 osd.7 42 *** Got signal Terminated *** 2023-06-23T18:17:40.064 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-06-23T18:17:40.048+0000 7fa2f24aa700 -1 osd.7 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.064 INFO:tasks.ceph.mds.b.smithi169.stderr:2023-06-23T18:17:40.048+0000 7faddecfa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 41469) UID: 0 2023-06-23T18:17:40.065 INFO:tasks.ceph.mds.b.smithi169.stderr:2023-06-23T18:17:40.048+0000 7faddecfa700 -1 mds.b *** got signal Terminated *** 2023-06-23T18:17:40.065 INFO:tasks.ceph.mds.d.smithi169.stderr:2023-06-23T18:17:40.052+0000 7f2eed179700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 41471) UID: 0 2023-06-23T18:17:40.065 INFO:tasks.ceph.mds.d.smithi169.stderr:2023-06-23T18:17:40.052+0000 7f2eed179700 -1 mds.d *** got signal Terminated *** 2023-06-23T18:17:40.066 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:17:40.052+0000 7eff7dfc3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 35457) UID: 0 2023-06-23T18:17:40.066 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:17:40.052+0000 7eff7dfc3700 -1 osd.5 42 *** Got signal Terminated *** 2023-06-23T18:17:40.066 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-06-23T18:17:40.052+0000 7eff7dfc3700 -1 osd.5 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.067 INFO:tasks.ceph.mon.b.smithi169.stderr:2023-06-23T18:17:40.052+0000 7fcacdc04700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 35118) UID: 0 2023-06-23T18:17:40.067 INFO:tasks.ceph.mon.b.smithi169.stderr:2023-06-23T18:17:40.052+0000 7fcacdc04700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-06-23T18:17:40.068 INFO:tasks.ceph.mon.c.smithi169.stderr:2023-06-23T18:17:40.052+0000 7feb74ac7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 35121) UID: 0 2023-06-23T18:17:40.068 INFO:tasks.ceph.mon.c.smithi169.stderr:2023-06-23T18:17:40.052+0000 7feb74ac7700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-06-23T18:17:40.069 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:17:40.039+0000 7f98b4e32700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35209) UID: 0 2023-06-23T18:17:40.070 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:17:40.039+0000 7f98b4e32700 -1 osd.2 42 *** Got signal Terminated *** 2023-06-23T18:17:40.070 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:17:40.039+0000 7fccacc02700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35215) UID: 0 2023-06-23T18:17:40.070 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:17:40.039+0000 7fccacc02700 -1 osd.3 42 *** Got signal Terminated *** 2023-06-23T18:17:40.071 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-06-23T18:17:40.039+0000 7fccacc02700 -1 osd.3 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.071 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-06-23T18:17:40.039+0000 7f98b4e32700 -1 osd.2 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.072 INFO:tasks.ceph.mds.a.smithi022.stderr:2023-06-23T18:17:40.043+0000 7f47ee6c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 46217) UID: 0 2023-06-23T18:17:40.072 INFO:tasks.ceph.mds.a.smithi022.stderr:2023-06-23T18:17:40.043+0000 7f47ee6c6700 -1 mds.a *** got signal Terminated *** 2023-06-23T18:17:40.072 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:17:40.039+0000 7fcb5e163700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35213) UID: 0 2023-06-23T18:17:40.073 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:17:40.043+0000 7fcb5e163700 -1 osd.1 42 *** Got signal Terminated *** 2023-06-23T18:17:40.073 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-06-23T18:17:40.043+0000 7fcb5e163700 -1 osd.1 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.073 INFO:tasks.ceph.mon.a.smithi022.stderr:2023-06-23T18:17:40.043+0000 7f9c160e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 35086) UID: 0 2023-06-23T18:17:40.073 INFO:tasks.ceph.mon.a.smithi022.stderr:2023-06-23T18:17:40.043+0000 7f9c160e9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-23T18:17:40.074 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:17:40.035+0000 7efd54dbc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35211) UID: 0 2023-06-23T18:17:40.074 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:17:40.035+0000 7efd54dbc700 -1 osd.0 42 *** Got signal Terminated *** 2023-06-23T18:17:40.074 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-06-23T18:17:40.035+0000 7efd54dbc700 -1 osd.0 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:17:40.263 INFO:tasks.ceph.mgr.x.smithi169.stderr:daemon-helper: command crashed with signal 15 2023-06-23T18:17:54.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:17:54.080 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:18:24.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:18:24.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:18:54.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:18:54.184 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:19:24.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:19:24.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:19:54.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:19:54.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:20:24.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:20:24.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:20:54.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:20:54.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:21:24.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:21:24.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:21:54.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:21:54.578 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:22:24.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:22:24.667 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:22:54.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:22:54.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:23:24.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:23:24.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:23:54.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:23:54.845 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:24:24.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:24:24.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:24:55.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:24:55.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:25:25.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:25:25.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:25:55.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:25:55.201 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:26:25.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:26:25.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:26:55.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:26:55.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:27:25.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:27:25.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:27:55.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:27:55.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:28:25.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:28:25.602 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:28:55.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:28:55.810 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:29:25.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:29:25.872 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:29:55.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:29:55.948 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:30:25.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:30:25.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:30:56.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:30:56.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:31:26.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:31:26.261 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:31:56.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:31:56.481 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:32:26.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:32:26.534 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:32:56.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:32:56.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:33:26.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:33:26.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:33:57.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:33:57.031 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:27.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:27.177 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:57.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:34:57.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:27.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:27.582 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:57.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:35:57.940 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:28.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:28.043 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:58.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:36:58.106 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:28.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:28.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:58.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:37:58.306 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:38:28.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:38:28.436 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:38:58.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:38:58.547 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:39:28.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:39:28.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:39:58.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:39:58.839 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:40:28.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:40:28.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:40:58.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:40:58.946 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:41:29.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:41:29.043 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:41:59.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:41:59.164 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:42:29.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:42:29.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:42:59.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:42:59.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:43:29.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:43:29.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:43:59.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:43:59.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:44:29.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:44:29.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:44:59.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:44:59.855 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:45:29.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:45:30.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:46:00.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:46:00.139 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:46:30.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:46:30.284 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:47:00.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:47:00.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:47:30.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:47:30.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:48:00.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:48:00.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:48:30.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:48:30.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:49:00.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:49:00.872 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:49:30.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:49:30.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:50:01.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:50:01.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:50:31.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:50:31.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:51:01.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:51:01.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:51:31.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:51:31.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:52:01.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:52:01.497 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:52:31.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:52:31.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:53:01.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:53:01.831 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:53:31.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:53:31.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:54:02.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:54:02.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:54:32.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:54:32.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:55:02.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:55:02.240 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:55:32.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:55:32.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:56:02.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:56:02.497 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:56:32.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:56:32.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:57:02.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:57:02.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:57:32.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:57:32.793 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:58:02.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:58:02.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:58:33.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:58:33.028 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:59:03.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:59:03.164 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:59:33.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T18:59:33.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:00:03.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:00:03.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:00:33.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:00:33.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:01:03.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:01:03.598 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:01:33.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:01:33.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:02:03.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:02:03.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:02:33.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:02:33.828 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:03:03.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:03:03.957 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:03:34.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:03:34.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:04:04.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:04:04.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:04:34.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:04:34.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:05:04.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:05:04.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:05:34.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:05:34.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:06:04.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:06:04.523 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:06:34.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:06:34.696 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:07:04.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:07:04.814 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:07:34.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:07:34.920 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:08:05.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:08:05.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:08:35.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:08:35.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:09:05.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:09:05.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:09:35.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:09:35.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:10:05.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:10:05.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:10:35.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:10:35.619 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:11:05.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:11:05.789 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:11:35.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:11:35.936 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:12:06.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:12:06.057 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:12:36.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:12:36.161 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:13:06.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:13:06.298 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:13:36.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:13:36.428 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:14:06.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:14:06.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:14:36.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:14:36.885 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:15:07.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:15:07.107 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:15:37.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:15:37.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:16:07.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:16:07.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:16:37.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:16:37.403 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:17:07.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:17:07.540 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:17:37.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:17:37.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:18:07.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:18:07.724 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:18:37.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:18:37.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:19:07.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:19:07.932 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:19:38.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:19:38.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:20:08.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:20:08.131 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:20:38.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:20:38.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:21:08.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:21:08.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:21:38.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:21:38.497 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:22:08.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:22:08.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:22:38.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:22:38.730 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:23:08.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:23:08.867 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:23:38.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:23:38.946 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:24:09.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:24:09.076 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:24:39.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:24:39.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:25:09.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:25:09.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:25:39.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:25:39.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:26:09.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:26:09.526 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:26:39.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:26:39.663 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:27:09.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:27:09.726 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:27:39.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:27:39.863 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:28:09.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:28:09.979 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:28:40.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:28:40.090 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:29:10.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:29:10.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:29:40.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:29:40.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:30:10.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:30:10.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:30:40.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:30:40.572 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:31:10.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:31:10.700 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:31:40.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:31:40.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:32:10.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:32:10.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:32:41.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:32:41.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:33:11.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:33:11.167 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:33:41.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:33:41.296 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:34:11.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:34:11.408 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:34:41.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:34:41.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:35:11.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:35:11.617 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:35:41.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:35:41.680 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:36:11.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:36:11.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:36:41.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:36:41.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:37:11.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:37:12.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:37:42.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:37:42.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:38:12.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:38:12.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:38:42.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:38:42.363 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:39:12.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:39:12.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:39:42.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:39:42.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:40:12.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:40:12.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:40:42.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:40:42.848 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:41:12.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:41:12.908 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:41:43.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:41:43.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:42:13.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:42:13.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:42:43.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:42:43.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:43:13.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:43:13.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:43:43.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:43:43.529 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:44:13.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:44:13.667 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:44:43.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:44:43.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:45:13.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:45:13.870 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:45:44.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:45:44.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:46:14.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:46:14.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:46:44.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:46:44.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:47:14.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:47:14.384 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:47:44.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:47:44.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:48:14.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:48:14.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:48:44.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:48:44.763 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:49:14.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:49:14.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:49:45.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:49:45.047 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:50:15.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:50:15.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:50:45.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:50:45.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:51:15.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:51:15.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:51:45.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:51:45.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:52:15.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:52:15.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:52:45.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:52:45.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:53:15.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:53:15.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:53:45.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:53:45.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:54:16.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:54:16.083 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:54:46.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:54:46.213 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:55:16.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:55:16.292 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:55:46.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:55:46.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:56:16.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:56:16.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:56:46.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:56:46.712 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:57:16.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:57:16.819 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:57:47.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:57:47.012 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:58:17.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:58:17.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:58:47.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:58:47.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:59:17.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:59:17.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:59:47.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T19:59:47.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:00:17.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:00:17.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:00:47.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:00:47.672 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:01:17.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:01:17.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:01:47.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:01:47.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:02:17.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:02:17.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:02:48.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:02:48.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:03:18.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:03:18.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:03:48.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:03:48.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:04:18.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:04:18.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:04:48.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:04:48.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:05:18.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:05:18.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:05:48.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:05:48.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:06:18.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:06:18.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:06:48.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:06:48.596 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:07:18.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:07:18.650 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:07:48.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:07:48.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:08:18.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:08:18.758 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:08:48.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:08:48.812 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:09:18.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:09:18.866 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:09:48.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:09:48.939 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:10:19.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:10:19.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:10:49.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:10:49.145 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:11:19.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:11:19.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:11:49.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:11:49.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:12:19.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:12:19.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:12:49.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:12:49.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:13:19.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:13:19.749 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:13:49.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:13:49.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:14:19.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:14:19.949 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:14:50.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:14:50.061 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:15:20.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:15:20.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:15:50.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:15:50.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:16:20.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:16:20.265 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:16:50.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:16:50.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:17:20.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:17:20.373 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:17:50.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:17:50.428 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:18:20.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:18:20.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:18:50.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:18:50.538 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:19:20.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:19:20.658 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:19:50.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:19:50.770 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:20:20.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:20:20.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:20:50.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:20:50.979 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:21:21.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:21:21.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:21:51.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:21:51.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:22:21.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:22:21.195 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:22:51.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:22:51.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:23:21.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:23:21.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:23:51.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:23:51.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:24:21.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:24:21.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:24:51.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:24:51.486 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:25:21.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:25:21.548 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:25:51.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:25:51.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:26:21.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:26:21.823 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:26:51.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:26:51.945 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:27:22.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:27:22.057 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:27:52.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:27:52.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:28:22.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:28:22.167 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:28:52.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:28:52.227 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:29:22.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:29:22.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:29:52.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:29:52.456 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:30:22.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:30:22.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:30:52.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:30:52.575 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:31:22.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:31:22.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:31:52.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:31:52.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:32:22.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:32:22.737 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:32:52.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:32:52.808 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:33:22.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:33:22.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:33:52.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:33:52.908 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:34:22.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:34:22.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:34:53.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:34:53.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:35:23.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:35:23.106 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:35:53.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:35:53.160 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:36:23.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:36:23.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:36:53.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:36:53.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:37:23.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:37:23.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:37:53.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:37:53.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:38:23.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:38:23.822 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:38:53.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:38:53.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:39:24.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:39:24.073 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:39:54.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:39:54.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:40:24.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:40:24.264 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:40:54.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:40:54.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:41:24.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:41:24.497 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:41:54.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:41:54.561 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:42:24.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:42:24.689 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:42:54.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:42:54.826 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:43:24.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:43:24.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:43:55.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:43:55.027 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:44:25.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:44:25.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:44:55.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:44:55.293 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:45:25.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:45:25.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:45:55.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:45:55.415 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:46:25.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:46:25.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:46:55.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:46:55.668 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:47:25.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:47:25.756 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:47:55.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:47:55.868 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:48:25.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:48:25.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:48:56.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:48:56.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:49:26.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:49:26.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:49:56.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:49:56.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:50:26.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:50:26.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:50:56.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:50:56.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:51:26.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:51:26.488 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:51:56.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:51:56.601 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:52:26.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:52:26.658 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:52:56.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:52:56.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:53:26.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:53:26.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:53:56.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:53:56.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:54:26.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:54:26.865 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:54:56.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:54:56.937 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:55:26.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:55:26.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:55:57.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:55:57.054 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:56:27.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:56:27.109 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:56:57.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:56:57.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:57:27.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:57:27.217 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:57:57.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:57:57.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:58:27.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:58:27.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:58:57.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:58:57.375 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:59:27.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:59:27.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:59:57.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T20:59:57.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:00:27.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:00:27.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:00:57.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:00:57.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:01:27.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:01:27.687 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:01:57.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:01:57.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:02:27.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:02:27.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:02:57.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:02:57.841 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:03:27.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:03:27.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:03:58.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:03:58.075 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:04:28.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:04:28.213 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:04:58.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:04:58.292 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:05:28.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:05:28.363 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:05:58.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:05:58.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:06:28.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:06:28.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:06:58.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:06:58.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:07:28.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:07:28.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:07:59.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:07:59.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:08:29.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:08:29.146 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:08:59.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:08:59.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:09:29.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:09:29.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:09:59.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:09:59.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:10:29.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:10:29.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:10:59.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:10:59.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:11:29.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:11:29.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:11:59.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:11:59.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:12:30.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:12:30.120 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:13:00.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:13:00.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:13:30.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:13:30.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:14:00.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:14:00.433 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:14:30.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:14:30.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:15:00.597 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:15:00.600 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:15:30.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:15:30.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:16:00.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:16:00.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:16:30.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:16:30.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:17:01.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:17:01.151 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:17:31.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:17:31.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:18:01.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:18:01.347 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:18:31.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:18:31.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:19:01.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:19:01.542 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:19:31.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:19:31.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:20:01.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:20:01.791 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:20:31.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:20:31.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:21:01.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:21:01.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:21:32.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:21:32.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:22:02.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:22:02.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:22:32.269 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:22:32.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:23:02.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:23:02.400 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:23:32.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:23:32.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:24:02.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:24:02.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:24:32.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:24:32.696 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:25:02.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:25:02.808 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:25:32.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:25:32.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:26:03.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:26:03.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:26:33.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:26:33.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:27:03.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:27:03.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:27:33.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:27:33.395 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:28:03.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:28:03.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:28:33.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:28:33.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:29:03.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:29:03.775 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:29:33.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:29:33.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:30:04.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:30:04.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:30:34.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:30:34.121 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:31:04.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:31:04.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:31:34.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:31:34.384 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:32:04.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:32:04.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:32:34.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:32:34.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:33:04.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:33:04.791 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:33:34.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:33:34.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:34:05.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:34:05.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:34:35.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:34:35.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:35:05.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:35:05.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:35:35.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:35:35.346 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:36:05.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:36:05.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:36:35.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:36:35.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:37:05.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:37:05.701 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:37:35.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:37:35.888 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:38:06.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:38:06.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:38:36.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:38:36.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:39:06.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:39:06.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:39:36.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:39:36.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:40:06.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:40:06.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:40:36.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:40:36.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:41:06.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:41:06.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:41:36.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:41:36.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:42:06.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:42:06.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:42:36.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:42:36.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:43:06.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:43:06.840 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:43:36.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:43:36.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:44:07.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:44:07.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:44:37.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:44:37.204 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:45:07.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:45:07.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:45:37.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:45:37.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:46:07.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:46:07.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:46:37.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:46:37.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:47:07.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:47:07.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:47:37.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:47:37.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:48:07.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:48:07.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:48:38.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:48:38.087 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:49:08.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:49:08.225 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:49:38.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:49:38.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:50:08.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:50:08.352 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:50:38.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:50:38.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:51:08.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:51:08.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:51:38.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:51:38.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:52:08.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:52:08.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:52:38.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:52:38.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:53:08.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:53:08.678 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:53:38.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:53:38.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:54:08.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:54:08.786 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:54:38.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:54:38.837 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:55:08.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:55:08.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:55:38.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:55:38.963 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:56:09.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:56:09.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:56:39.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:56:39.079 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:57:09.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:57:09.138 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:57:39.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:57:39.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:58:09.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:58:09.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:58:39.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:58:39.296 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:59:09.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:59:09.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:59:39.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T21:59:39.404 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:00:09.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:00:09.480 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:00:39.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:00:39.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:01:09.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:01:09.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:01:39.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:01:39.645 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:02:09.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:02:09.698 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:02:39.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:02:39.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:03:09.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:03:09.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:03:39.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:03:39.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:04:09.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:04:09.914 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:04:39.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:04:39.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:05:10.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:05:10.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:05:40.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:05:40.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:06:10.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:06:10.156 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:06:40.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:06:40.202 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:07:10.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:07:10.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:07:40.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:07:40.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:08:10.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:08:10.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:08:40.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:08:40.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:09:10.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:09:10.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:09:40.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:09:40.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:10:10.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:10:10.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:10:41.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:10:41.062 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:11:11.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:11:11.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:11:41.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:11:41.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:12:11.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:12:11.243 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:12:41.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:12:41.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:13:11.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:13:11.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:13:41.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:13:41.394 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:14:11.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:14:11.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:14:41.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:14:41.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:15:11.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:15:11.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:15:41.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:15:41.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:16:11.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:16:11.770 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:16:41.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:16:41.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:17:11.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:17:11.949 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:17:42.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:17:42.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:18:12.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:18:12.216 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:18:42.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:18:42.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:19:12.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:19:12.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:19:42.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:19:42.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:20:12.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:20:12.674 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:20:42.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:20:42.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:21:12.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:21:12.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:21:42.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:21:42.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:22:13.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:22:13.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:22:43.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:22:43.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:23:13.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:23:13.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:23:43.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:23:43.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:24:13.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:24:13.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:24:43.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:24:43.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:25:13.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:25:13.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:25:43.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:25:43.912 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:26:14.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:26:14.017 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:26:44.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:26:44.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:27:14.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:27:14.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:27:44.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:27:44.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:28:14.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:28:14.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:28:44.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:28:44.417 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:29:14.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:29:14.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:29:44.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:29:44.521 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:30:14.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:30:14.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:30:44.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:30:44.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:31:14.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:31:14.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:31:44.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:31:44.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:32:14.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:32:14.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:32:45.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:32:45.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:33:15.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:33:15.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:33:45.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:33:45.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:34:15.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:34:15.479 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:34:45.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:34:45.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:35:15.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:35:15.747 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:35:45.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:35:45.810 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:36:15.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:36:15.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:36:45.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:36:45.917 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:37:15.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:37:15.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:37:46.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:37:46.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:38:16.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:38:16.081 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:38:46.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:38:46.133 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:39:16.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:39:16.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:39:46.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:39:46.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:40:16.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:40:16.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:40:46.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:40:46.384 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:41:16.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:41:16.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:41:46.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:41:46.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:42:16.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:42:16.553 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:42:46.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:42:46.598 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:43:16.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:43:16.712 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:43:46.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:43:46.900 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:44:16.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:44:16.954 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:44:47.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:44:47.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:45:17.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:45:17.163 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:45:47.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:45:47.234 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:46:17.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:46:17.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:46:47.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:46:47.350 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:47:17.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:47:17.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:47:47.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:47:47.467 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:48:17.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:48:17.521 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:48:47.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:48:47.568 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:49:17.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:49:17.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:49:47.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:49:47.674 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:50:17.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:50:17.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:50:47.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:50:47.808 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:51:17.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:51:17.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:51:47.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:51:47.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:52:17.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:52:17.956 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:52:48.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:52:48.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:53:18.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:53:18.070 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:53:48.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:53:48.116 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:54:18.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:54:18.175 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:54:48.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:54:48.224 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:55:18.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:55:18.284 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:55:48.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:55:48.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:56:18.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:56:18.417 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:56:48.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:56:48.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:57:18.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:57:18.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:57:48.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:57:48.574 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:58:18.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:58:18.634 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:58:48.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:58:48.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:59:18.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:59:18.742 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:59:48.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T22:59:48.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:00:18.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:00:18.869 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:00:48.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:00:48.940 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:01:18.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:01:19.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:01:49.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:01:49.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:02:19.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:02:19.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:02:49.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:02:49.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:03:19.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:03:19.235 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:03:49.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:03:49.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:04:19.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:04:19.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:04:49.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:04:49.397 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:05:19.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:05:19.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:05:49.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:05:49.547 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:06:19.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:06:19.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:06:49.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:06:49.780 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:07:19.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:07:19.894 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:07:50.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:07:50.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:08:20.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:08:20.151 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:08:50.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:08:50.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:09:20.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:09:20.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:09:50.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:09:50.556 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:10:20.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:10:20.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:10:50.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:10:50.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:11:20.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:11:20.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:11:50.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:11:50.988 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:12:21.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:12:21.093 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:12:51.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:12:51.213 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:13:21.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:13:21.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:13:51.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:13:51.455 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:14:21.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:14:21.567 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:14:51.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:14:51.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:15:21.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:15:21.725 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:15:51.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:15:51.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:16:21.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:16:21.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:16:52.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:16:52.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:17:22.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:17:22.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:17:52.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:17:52.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:18:22.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:18:22.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:18:52.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:18:52.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:19:22.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:19:22.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:19:52.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:19:52.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:20:22.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:20:22.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:20:52.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:20:52.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:21:22.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:21:23.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:21:53.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:21:53.138 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:22:23.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:22:23.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:22:53.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:22:53.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:23:23.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:23:23.415 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:23:53.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:23:53.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:24:23.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:24:23.674 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:24:53.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:24:53.787 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:25:23.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:25:23.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:25:53.975 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:25:53.978 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:26:24.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:26:24.082 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:26:54.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:26:54.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:27:24.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:27:24.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:27:54.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:27:54.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:28:24.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:28:24.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:28:54.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:28:54.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:29:24.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:29:24.688 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:29:54.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:29:54.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:30:24.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:30:24.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:30:54.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:30:55.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:31:25.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:31:25.063 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:31:55.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:31:55.109 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:32:25.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:32:25.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:32:55.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:32:55.227 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:33:25.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:33:25.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:33:55.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:33:55.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:34:25.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:34:25.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:34:55.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:34:55.442 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:35:25.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:35:25.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:35:55.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:35:55.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:36:25.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:36:25.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:36:55.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:36:55.817 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:37:25.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:37:25.872 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:37:55.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:37:55.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:38:26.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:38:26.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:38:56.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:38:56.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:39:26.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:39:26.436 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:39:56.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:39:56.676 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:40:26.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:40:26.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:40:56.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:40:56.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:41:26.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:41:26.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:41:57.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:41:57.125 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:42:27.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:42:27.254 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:42:57.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:42:57.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:43:27.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:43:27.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:43:57.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:43:57.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:44:27.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:44:27.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:44:57.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:44:57.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:45:27.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:45:27.640 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:45:57.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:45:57.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:46:27.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:46:27.766 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:46:57.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:46:57.809 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:47:27.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:47:27.870 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:47:57.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:47:57.918 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:48:27.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:48:27.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:48:58.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:48:58.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:49:28.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:49:28.080 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:49:58.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:49:58.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:50:28.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:50:28.205 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:50:58.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:50:58.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:51:28.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:51:28.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:51:58.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:51:58.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:52:28.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:52:28.440 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:52:58.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:52:58.483 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:53:28.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:53:28.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:53:58.597 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:53:58.600 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:54:28.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:54:28.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:54:58.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:54:58.707 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:55:28.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:55:28.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:55:58.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:55:58.841 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:56:28.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:56:28.893 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:56:58.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:56:58.942 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:57:28.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:57:28.998 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:57:59.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:57:59.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:58:29.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:58:29.105 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:58:59.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:58:59.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:59:29.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:59:29.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:59:59.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T23:59:59.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:00:29.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:00:29.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:00:59.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:00:59.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:01:29.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:01:29.460 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:01:59.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:01:59.506 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:02:29.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:02:29.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:02:59.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:02:59.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:03:29.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:03:29.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:03:59.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:03:59.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:04:30.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:04:30.031 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:05:00.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:05:00.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:05:30.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:05:30.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:06:00.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:06:00.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:06:30.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:06:30.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:07:00.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:07:00.944 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:07:30.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:07:31.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:08:01.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:08:01.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:08:31.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:08:31.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:09:01.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:09:01.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:09:31.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:09:31.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:10:01.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:10:01.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:10:31.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:10:31.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:11:02.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:11:02.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:11:32.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:11:32.106 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:12:02.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:12:02.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:12:32.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:12:32.357 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:13:02.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:13:02.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:13:32.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:13:32.581 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:14:02.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:14:02.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:14:32.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:14:32.841 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:15:02.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:15:02.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:15:33.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:15:33.107 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:16:03.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:16:03.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:16:33.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:16:33.238 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:17:03.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:17:03.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:17:33.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:17:33.499 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:18:03.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:18:03.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:18:33.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:18:33.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:19:03.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:19:03.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:19:33.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:19:33.974 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:20:04.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:20:04.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:20:34.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:20:34.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:21:04.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:21:04.327 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:21:34.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:21:34.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:22:04.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:22:04.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:22:34.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:22:34.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:23:04.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:23:04.878 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:23:34.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:23:34.990 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:24:05.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:24:05.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:24:35.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:24:35.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:25:05.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:25:05.453 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:25:35.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:25:35.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:26:05.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:26:05.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:26:35.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:26:35.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:27:05.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:27:05.794 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:27:35.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:27:35.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:28:05.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:28:05.917 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:28:35.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:28:35.989 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:29:06.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:29:06.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:29:36.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:29:36.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:30:06.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:30:06.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:30:36.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:30:36.230 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:31:06.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:31:06.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:31:36.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:31:36.365 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:32:06.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:32:06.410 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:32:36.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:32:36.472 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:33:06.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:33:06.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:33:36.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:33:36.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:34:06.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:34:06.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:34:36.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:34:36.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:35:06.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:35:06.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:35:36.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:35:36.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:36:06.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:36:06.895 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:36:36.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:36:36.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:37:07.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:37:07.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:37:37.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:37:37.079 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:38:07.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:38:07.136 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:38:37.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:38:37.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:39:07.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:39:07.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:39:37.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:39:37.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:40:07.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:40:07.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:40:37.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:40:37.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:41:07.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:41:07.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:41:37.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:41:37.873 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:42:07.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:42:07.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:42:37.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:42:37.984 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:43:08.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:43:08.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:43:38.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:43:38.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:44:08.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:44:08.153 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:44:38.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:44:38.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:45:08.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:45:08.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:45:38.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:45:38.541 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:46:08.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:46:08.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:46:38.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:46:38.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:47:08.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:47:08.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:47:38.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:47:38.990 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:48:09.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:48:09.111 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:48:39.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:48:39.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:49:09.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:49:09.363 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:49:39.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:49:39.457 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:50:09.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:50:09.586 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:50:39.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:50:39.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:51:09.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:51:09.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:51:39.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:51:39.932 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:52:10.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:52:10.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:52:40.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:52:40.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:53:10.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:53:10.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:53:40.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:53:40.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:54:10.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:54:10.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:54:40.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:54:40.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:55:10.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:55:10.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:55:40.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:55:40.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:56:11.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:56:11.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:56:41.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:56:41.199 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:57:11.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:57:11.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:57:41.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:57:41.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:58:11.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:58:11.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:58:41.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:58:41.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:59:11.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:59:11.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:59:41.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T00:59:41.931 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:00:12.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:00:12.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:00:42.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:00:42.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:01:12.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:01:12.228 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:01:42.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:01:42.357 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:02:12.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:02:12.504 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:02:42.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:02:42.680 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:03:12.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:03:12.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:03:42.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:03:42.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:04:13.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:04:13.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:04:43.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:04:43.217 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:05:13.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:05:13.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:05:43.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:05:43.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:06:13.552 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:06:13.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:06:43.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:06:43.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:07:13.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:07:13.872 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:07:43.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:07:44.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:08:14.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:08:14.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:08:44.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:08:44.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:09:14.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:09:14.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:09:44.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:09:44.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:10:14.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:10:14.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:10:44.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:10:44.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:11:14.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:11:14.855 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:11:44.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:11:44.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:12:15.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:12:15.079 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:12:45.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:12:45.199 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:13:15.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:13:15.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:13:45.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:13:45.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:14:15.534 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:14:15.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:14:45.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:14:45.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:15:15.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:15:15.787 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:15:45.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:15:45.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:16:15.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:16:15.987 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:16:46.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:16:46.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:17:16.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:17:16.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:17:46.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:17:46.391 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:18:16.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:18:16.504 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:18:46.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:18:46.574 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:19:16.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:19:16.632 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:19:46.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:19:46.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:20:16.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:20:16.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:20:46.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:20:46.874 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:21:16.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:21:16.945 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:21:47.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:21:47.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:22:17.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:22:17.054 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:22:47.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:22:47.116 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:23:17.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:23:17.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:23:47.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:23:47.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:24:17.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:24:17.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:24:47.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:24:47.357 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:25:17.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:25:17.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:25:47.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:25:47.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:26:17.534 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:26:17.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:26:47.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:26:47.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:27:17.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:27:17.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:27:48.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:27:48.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:28:18.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:28:18.244 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:28:48.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:28:48.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:29:18.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:29:18.494 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:29:48.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:29:48.606 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:30:18.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:30:18.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:30:48.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:30:48.889 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:31:18.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:31:18.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:31:49.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:31:49.022 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:32:19.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:32:19.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:32:49.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:32:49.144 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:33:19.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:33:19.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:33:49.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:33:49.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:34:19.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:34:19.317 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:34:49.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:34:49.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:35:19.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:35:19.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:35:49.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:35:49.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:36:19.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:36:19.634 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:36:49.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:36:49.697 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:37:19.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:37:19.760 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:37:49.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:37:49.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:38:19.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:38:19.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:38:49.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:38:49.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:39:19.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:39:19.991 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:39:50.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:39:50.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:40:20.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:40:20.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:40:50.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:40:50.379 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:41:20.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:41:20.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:41:50.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:41:50.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:42:20.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:42:20.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:42:50.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:42:50.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:43:20.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:43:20.983 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:43:51.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:43:51.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:44:21.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:44:21.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:44:51.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:44:51.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:45:21.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:45:21.424 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:45:51.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:45:51.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:46:21.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:46:21.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:46:51.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:46:51.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:47:21.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:47:21.684 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:47:51.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:47:51.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:48:21.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:48:21.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:48:51.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:48:51.873 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:49:21.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:49:21.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:49:51.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:49:51.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:50:22.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:50:22.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:50:52.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:50:52.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:51:22.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:51:22.298 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:51:52.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:51:52.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:52:22.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:52:22.489 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:52:52.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:52:52.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:53:22.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:53:22.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:53:52.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:53:52.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:54:22.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:54:22.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:54:53.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:54:53.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:55:23.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:55:23.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:55:53.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:55:53.261 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:56:23.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:56:23.331 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:56:53.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:56:53.459 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:57:23.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:57:23.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:57:53.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:57:53.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:58:23.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:58:23.814 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:58:53.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:58:53.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:59:24.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:59:24.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:59:54.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T01:59:54.096 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:00:24.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:00:24.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:00:54.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:00:54.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:01:24.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:01:24.404 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:01:54.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:01:54.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:02:24.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:02:24.679 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:02:54.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:02:54.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:03:24.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:03:24.928 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:03:55.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:03:55.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:04:25.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:04:25.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:04:55.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:04:55.507 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:05:25.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:05:25.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:05:55.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:05:55.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:06:25.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:06:25.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:06:56.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:06:56.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:07:26.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:07:26.105 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:07:56.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:07:56.216 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:08:26.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:08:26.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:08:56.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:08:56.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:09:26.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:09:26.461 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:09:56.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:09:56.607 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:10:26.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:10:26.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:10:56.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:10:56.782 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:11:27.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:11:27.111 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:11:57.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:11:57.223 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:12:27.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:12:27.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:12:57.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:12:57.416 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:13:27.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:13:27.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:13:57.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:13:57.658 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:14:27.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:14:27.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:14:57.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:14:57.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:15:28.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:15:28.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:15:58.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:15:58.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:16:28.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:16:28.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:16:58.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:16:58.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:17:28.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:17:28.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:17:58.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:17:58.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:18:28.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:18:28.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:18:58.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:18:58.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:19:28.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:19:28.804 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:19:58.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:19:58.867 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:20:28.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:20:29.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:20:59.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:20:59.083 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:21:29.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:21:29.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:21:59.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:21:59.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:22:29.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:22:29.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:22:59.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:22:59.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:23:29.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:23:29.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:23:59.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:23:59.788 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:24:29.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:24:29.901 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:24:59.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:24:59.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:25:30.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:25:30.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:26:00.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:26:00.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:26:30.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:26:30.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:27:00.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:27:00.463 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:27:30.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:27:30.592 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:28:00.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:28:00.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:28:30.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:28:30.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:29:00.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:29:00.998 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:29:31.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:29:31.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:30:01.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:30:01.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:30:31.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:30:31.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:31:01.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:31:01.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:31:31.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:31:31.488 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:32:01.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:32:01.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:32:31.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:32:31.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:33:02.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:33:02.056 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:33:32.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:33:32.161 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:34:02.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:34:02.306 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:34:32.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:34:32.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:35:02.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:35:02.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:35:32.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:35:32.643 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:36:02.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:36:02.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:36:32.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:36:32.826 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:37:02.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:37:02.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:37:32.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:37:32.947 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:38:03.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:38:03.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:38:33.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:38:33.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:39:03.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:39:03.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:39:33.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:39:33.202 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:40:03.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:40:03.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:40:33.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:40:33.459 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:41:03.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:41:03.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:41:33.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:41:33.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:42:03.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:42:03.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:42:33.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:42:33.900 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:43:04.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:43:04.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:43:34.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:43:34.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:44:04.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:44:04.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:44:34.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:44:34.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:45:04.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:45:04.536 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:45:34.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:45:34.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:46:04.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:46:04.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:46:34.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:46:34.726 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:47:04.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:47:04.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:47:34.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:47:34.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:48:04.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:48:04.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:48:34.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:48:34.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:49:05.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:49:05.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:49:35.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:49:35.070 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:50:05.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:50:05.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:50:35.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:50:35.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:51:05.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:51:05.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:51:35.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:51:35.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:52:05.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:52:05.391 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:52:35.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:52:35.463 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:53:05.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:53:05.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:53:35.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:53:35.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:54:05.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:54:05.634 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:54:35.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:54:35.693 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:55:05.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:55:05.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:55:35.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:55:35.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:56:05.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:56:05.872 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:56:35.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:56:35.943 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:57:05.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:57:05.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:57:36.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:57:36.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:58:06.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:58:06.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:58:36.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:58:36.168 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:59:06.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:59:06.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:59:36.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T02:59:36.290 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:00:06.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:00:06.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:00:36.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:00:36.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:01:06.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:01:06.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:01:36.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:01:36.543 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:02:06.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:02:06.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:02:36.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:02:36.667 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:03:06.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:03:06.730 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:03:36.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:03:36.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:04:06.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:04:06.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:04:36.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:04:36.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:05:06.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:05:06.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:05:37.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:05:37.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:06:07.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:06:07.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:06:37.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:06:37.175 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:07:07.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:07:07.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:07:37.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:07:37.300 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:08:07.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:08:07.362 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:08:37.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:08:37.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:09:07.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:09:07.483 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:09:37.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:09:37.543 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:10:07.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:10:07.608 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:10:37.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:10:37.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:11:07.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:11:07.726 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:11:37.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:11:37.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:12:07.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:12:07.897 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:12:38.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:12:38.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:13:08.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:13:08.145 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:13:38.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:13:38.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:14:08.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:14:08.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:14:38.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:14:38.578 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:15:08.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:15:08.712 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:15:38.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:15:38.983 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:16:09.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:16:09.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:16:39.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:16:39.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:17:09.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:17:09.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:17:39.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:17:39.408 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:18:09.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:18:09.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:18:39.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:18:39.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:19:09.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:19:09.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:19:39.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:19:39.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:20:10.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:20:10.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:20:40.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:20:40.149 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:21:10.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:21:10.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:21:40.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:21:40.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:22:10.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:22:10.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:22:40.473 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:22:40.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:23:10.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:23:10.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:23:40.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:23:40.599 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:24:10.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:24:10.661 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:24:40.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:24:40.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:25:10.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:25:10.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:25:40.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:25:40.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:26:10.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:26:10.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:26:40.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:26:40.982 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:27:11.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:27:11.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:27:41.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:27:41.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:28:11.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:28:11.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:28:41.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:28:41.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:29:11.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:29:11.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:29:41.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:29:41.415 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:30:11.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:30:11.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:30:41.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:30:41.531 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:31:11.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:31:11.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:31:41.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:31:41.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:32:11.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:32:11.869 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:32:41.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:32:41.990 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:33:12.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:33:12.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:33:42.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:33:42.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:34:12.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:34:12.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:34:42.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:34:42.481 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:35:12.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:35:12.587 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:35:42.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:35:42.722 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:36:12.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:36:12.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:36:42.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:36:42.945 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:37:13.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:37:13.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:37:43.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:37:43.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:38:13.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:38:13.317 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:38:43.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:38:43.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:39:13.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:39:13.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:39:43.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:39:43.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:40:13.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:40:13.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:40:43.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:40:43.909 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:41:14.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:41:14.048 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:41:44.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:41:44.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:42:14.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:42:14.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:42:44.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:42:44.362 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:43:14.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:43:14.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:43:44.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:43:44.632 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:44:14.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:44:14.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:44:44.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:44:44.876 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:45:15.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:45:15.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:45:45.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:45:45.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:46:15.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:46:15.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:46:45.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:46:45.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:47:15.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:47:15.369 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:47:45.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:47:45.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:48:15.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:48:15.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:48:45.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:48:45.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:49:15.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:49:15.880 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:49:46.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:49:46.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:50:16.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:50:16.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:50:46.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:50:46.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:51:16.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:51:16.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:51:46.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:51:46.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:52:16.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:52:16.590 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:52:46.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:52:46.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:53:16.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:53:16.867 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:53:46.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:53:46.979 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:54:17.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:54:17.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:54:47.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:54:47.231 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:55:17.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:55:17.361 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:55:47.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:55:47.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:56:17.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:56:17.604 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:56:47.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:56:47.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:57:17.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:57:17.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:57:47.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:57:47.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:58:18.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:58:18.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:58:48.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:58:48.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:59:18.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:59:18.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:59:48.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T03:59:48.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:00:18.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:00:18.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:00:48.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:00:48.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:01:18.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:01:18.816 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:01:48.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:01:48.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:02:18.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:02:18.938 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:02:48.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:02:48.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:03:19.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:03:19.063 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:03:49.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:03:49.117 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:04:19.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:04:19.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:04:49.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:04:49.234 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:05:19.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:05:19.296 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:05:49.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:05:49.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:06:19.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:06:19.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:06:49.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:06:49.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:07:19.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:07:19.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:07:49.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:07:49.702 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:08:19.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:08:19.770 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:08:49.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:08:49.824 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:09:19.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:09:19.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:09:50.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:09:50.004 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:10:20.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:10:20.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:10:50.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:10:50.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:11:20.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:11:20.393 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:11:50.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:11:50.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:12:20.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:12:20.536 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:12:50.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:12:50.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:13:20.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:13:20.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:13:50.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:13:50.951 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:14:21.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:14:21.063 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:14:51.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:14:51.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:15:21.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:15:21.332 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:15:51.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:15:51.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:16:21.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:16:21.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:16:51.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:16:51.624 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:17:21.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:17:21.736 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:17:51.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:17:51.791 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:18:21.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:18:21.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:18:51.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:18:51.915 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:19:21.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:19:21.978 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:19:52.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:19:52.033 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:20:22.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:20:22.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:20:52.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:20:52.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:21:22.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:21:22.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:21:52.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:21:52.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:22:22.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:22:22.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:22:52.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:22:52.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:23:22.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:23:22.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:23:52.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:23:52.862 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:24:23.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:24:23.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:24:53.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:24:53.131 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:25:23.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:25:23.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:25:53.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:25:53.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:26:23.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:26:23.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:26:53.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:26:53.541 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:27:23.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:27:23.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:27:53.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:27:53.773 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:28:23.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:28:23.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:28:54.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:28:54.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:29:24.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:29:24.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:29:54.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:29:54.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:30:24.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:30:24.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:30:54.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:30:54.602 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:31:24.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:31:24.724 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:31:54.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:31:54.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:32:25.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:32:25.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:32:55.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:32:55.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:33:25.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:33:25.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:33:55.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:33:55.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:34:25.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:34:25.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:34:55.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:34:55.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:35:25.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:35:25.637 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:35:55.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:35:55.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:36:25.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:36:25.843 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:36:55.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:36:55.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:37:25.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:37:25.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:37:56.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:37:56.085 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:38:26.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:38:26.151 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:38:56.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:38:56.213 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:39:26.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:39:26.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:39:56.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:39:56.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:40:26.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:40:26.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:40:56.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:40:56.586 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:41:26.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:41:26.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:41:56.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:41:56.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:42:27.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:42:27.030 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:42:57.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:42:57.077 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:43:27.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:43:27.199 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:43:57.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:43:57.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:44:27.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:44:27.453 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:44:57.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:44:57.614 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:45:27.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:45:27.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:45:57.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:45:57.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:46:28.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:46:28.034 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:46:58.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:46:58.290 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:47:28.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:47:28.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:47:58.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:47:58.604 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:48:28.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:48:28.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:48:58.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:48:58.895 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:49:29.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:49:29.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:49:59.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:49:59.156 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:50:29.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:50:29.277 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:50:59.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:50:59.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:51:29.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:51:29.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:51:59.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:51:59.730 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:52:29.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:52:29.878 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:52:59.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:53:00.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:53:30.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:53:30.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:54:00.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:54:00.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:54:30.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:54:30.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:55:00.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:55:00.536 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:55:30.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:55:30.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:56:00.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:56:00.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:56:30.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:56:30.915 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:57:00.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:57:00.989 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:57:31.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:57:31.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:58:01.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:58:01.110 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:58:31.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:58:31.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:59:01.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:59:01.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:59:31.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T04:59:31.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:00:01.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:00:01.372 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:00:31.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:00:31.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:01:01.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:01:01.497 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:01:31.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:01:31.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:02:01.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:02:01.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:02:31.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:02:31.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:03:01.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:03:01.760 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:03:31.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:03:31.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:04:01.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:04:01.881 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:04:31.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:04:31.943 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:05:01.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:05:01.999 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:05:32.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:05:32.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:06:02.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:06:02.116 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:06:32.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:06:32.195 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:07:02.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:07:02.262 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:07:32.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:07:32.327 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:08:02.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:08:02.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:08:32.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:08:32.454 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:09:02.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:09:02.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:09:32.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:09:32.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:10:02.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:10:02.625 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:10:32.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:10:32.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:11:02.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:11:02.742 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:11:32.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:11:32.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:12:02.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:12:02.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:12:32.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:12:32.948 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:13:02.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:13:02.998 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:13:33.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:13:33.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:14:03.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:14:03.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:14:33.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:14:33.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:15:03.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:15:03.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:15:33.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:15:33.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:16:03.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:16:03.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:16:33.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:16:33.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:17:03.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:17:03.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:17:33.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:17:33.581 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:18:03.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:18:03.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:18:33.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:18:33.698 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:19:03.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:19:03.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:19:33.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:19:33.813 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:20:03.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:20:03.868 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:20:33.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:20:33.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:21:03.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:21:03.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:21:34.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:21:34.054 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:22:04.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:22:04.116 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:22:34.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:22:34.182 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:23:04.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:23:04.238 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:23:34.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:23:34.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:24:04.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:24:04.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:24:34.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:24:34.424 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:25:04.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:25:04.478 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:25:34.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:25:34.542 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:26:04.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:26:04.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:26:34.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:26:34.663 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:27:04.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:27:04.750 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:27:34.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:27:34.817 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:28:04.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:28:04.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:28:34.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:28:34.933 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:29:04.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:29:04.986 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:29:35.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:29:35.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:30:05.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:30:05.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:30:35.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:30:35.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:31:05.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:31:05.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:31:35.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:31:35.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:32:05.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:32:05.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:32:35.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:32:35.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:33:05.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:33:05.460 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:33:35.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:33:35.515 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:34:05.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:34:05.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:34:35.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:34:35.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:35:05.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:35:05.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:35:35.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:35:35.734 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:36:05.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:36:05.795 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:36:35.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:36:35.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:37:05.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:37:05.932 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:37:35.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:37:35.987 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:38:06.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:38:06.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:38:36.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:38:36.106 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:39:06.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:39:06.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:39:36.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:39:36.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:40:06.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:40:06.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:40:36.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:40:36.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:41:06.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:41:06.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:41:36.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:41:36.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:42:06.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:42:06.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:42:36.552 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:42:36.555 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:43:06.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:43:06.610 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:43:36.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:43:36.668 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:44:06.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:44:06.725 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:44:36.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:44:36.779 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:45:06.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:45:06.836 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:45:36.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:45:36.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:46:06.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:46:06.939 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:46:37.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:46:37.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:47:07.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:47:07.066 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:47:37.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:47:37.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:48:07.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:48:07.175 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:48:37.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:48:37.231 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:49:07.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:49:07.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:49:37.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:49:37.326 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:50:07.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:50:07.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:50:37.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:50:37.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:51:07.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:51:07.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:51:37.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:51:37.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-24T05:51:40.504 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-06-24T05:51:40.506 DEBUG:teuthology.task.console_log:Killing console logger for smithi022 2023-06-24T05:51:40.506 DEBUG:teuthology.task.console_log:Killing console logger for smithi169 2023-06-24T05:51:40.507 DEBUG:teuthology.exit:Finished running handlers